tokevenWhy caching matters
Cache-read tokens cost ~10% of input tokens. If you structure prompts to maximize cache hits, you can cut input costs by 90%.
How to improve cache rates
1. Put stable content first
System prompts, instructions, and context that doesn't change between calls should go at the beginning of your prompt. The model caches from the start of the prompt.
2. Reuse conversation prefixes
If multiple calls share the same context (e.g., a code review session), structure them so the shared prefix is identical.
3. Batch similar requests
Group similar tasks together in the same session. The shared context gets cached once and reused.
Tracking your progress
The Token Metrics dashboard shows your cache hit rate over time. Aim for 30%+ for meaningful savings.