Prompt caching saves 90%: a practical guide

The economics of caching

Cache-read tokens cost a fraction of regular input tokens — roughly 10% on Anthropic and OpenAI, and about 25% on Google (a 75% discount). If your system prompt is 3,000 tokens and you send it 100 times, caching saves you the bulk of that repeated input cost.

How prompt caching works

Providers cache the prefix of your prompt. If the next request starts with the same sequence of tokens, those tokens are read from cache instead of re-processed.

The key insight: order matters. Static content must come first. Variable content goes at the end.

Structuring prompts for cache hits

• -System prompt first — your instructions, persona, and rules
• -Reference material second — documentation, schemas, examples
• -Conversation history third — previous turns
• -Current query last — the variable part

Common mistakes

• -Putting a timestamp or request ID at the start of the prompt (invalidates the entire cache)
• -Randomizing the order of few-shot examples
• -Rebuilding the full prompt from scratch each call instead of appending

Measuring your cache rate

Track the ratio of cache-read tokens to total input tokens. A well-structured prompt flow should hit 70-90% cache rates on subsequent calls. If you're below 50%, your prompt structure needs work.

The Tokeven dashboard shows cache hit rate per session and per developer, so you can spot who's getting the benefit and who isn't.