Prompt caching: a practical guide

The economics of caching

Cache-read tokens cost a fraction of regular input tokens - 0.10x the input price, a 90% discount, on current OpenAI and Google models and on most Anthropic ones. Older OpenAI and Google models discount less or not at all, so across each catalogue OpenAI ranges 0-95% off and Google 0-90% off. Two families vary by model even among current models: Anthropic ranges 90-98% off (Claude Fable 5.1 goes furthest, at 98%), and xAI discounts less, at 75-85% off. Check the row rather than the brand on both. If your system prompt is 3,000 tokens and you send it 100 times, caching saves you the bulk of that repeated input cost.

Writing to cache is the part that varies. Anthropic charges a 25% premium on cache-write tokens, OpenAI charges it on the GPT 5.6 family only, and Google and xAI charge nothing. That fee lands on your misses, so it matters in proportion to how often your prefix changes.

How prompt caching works

Providers cache the prefix of your prompt. If the next request starts with the same sequence of tokens, those tokens are read from cache instead of re-processed.

The key insight: order matters. Static content must come first. Variable content goes at the end.

Structuring prompts for cache hits

• -System prompt first - your instructions, persona, and rules
• -Reference material second - documentation, schemas, examples
• -Conversation history third - previous turns
• -Current query last - the variable part

Common mistakes

• -Putting a timestamp or request ID at the start of the prompt (invalidates the entire cache)
• -Randomizing the order of few-shot examples
• -Rebuilding the full prompt from scratch each call instead of appending

Measuring your cache rate

Track the ratio of cache-read tokens to total input tokens. A well-structured prompt flow should hit 70-90% cache rates on subsequent calls. If you're below 50%, your prompt structure needs work.

The Tokens tab on Tokeven's Spend dashboard shows cache-read and cache-write tokens separately from fresh input, so you can see whether your caching is actually paying off.