LLM Prompt Caching Guide for Claude & GPT APIs
Keep system instructions, tool schemas, and early conversation prefixes deterministic so repeated agent turns bill at discounted cache_read rates instead of full input rates.
Why Prompt Caching Matters for Coding & RAG Agents
Coding agents and RAG pipelines resend thousands of system prompt and tool schema tokens every turn. When requests match a cached prefix, read tokens are billed at substantially lower cache_read rates (see Live Pricing).
3 Rules to Maximize Cache Hit Rates
- Move dynamic values to the end: Never prepend timestamps, request IDs, or per-turn metadata to your
systemblock. - Keep
toolsordering deterministic: Re-sorting tool definitions between turns invalidates the prefix cache. - Audit in Usage Logs: Verify
cache_readvscache_writetokens per request inside the DeepClub console usage logs.
Frequently Asked Questions
- Why am I seeing cache_write charges without cache_read hits?
- Usually a dynamic timestamp, random ID, or re-ordered tool schema changed near the beginning of your prompt prefix, forcing a fresh cache write each turn.
- Does prompt caching discount output tokens too?
- No. Prompt caching applies to cached input prefixes; generated output tokens are billed at the standard output rate listed on /pricing.
Updated