Token Metrics explained

Key metrics

• •Input tokens - tokens sent to the model (your prompts + context)
• •Output tokens - tokens generated by the model
• •Cache-read tokens - input tokens served from cache (~90% cheaper)
• •Cache-write tokens - input tokens written to cache (~25% more expensive, but pay off quickly)

Cache hit rate

Your cache hit rate shows what percentage of input tokens were served from cache. Higher is better.

• •0-10% - no caching in place, significant savings available
• •10-30% - some natural caching from repeated system prompts
• •30%+ - active caching strategy paying off

Improving your metrics

1. Structure prompts for cache hits - put stable content (system prompts, instructions) at the beginning
2. Reuse conversation prefixes - cache-friendly patterns save 90% on repeated context
3. Trim conversation context - summarize prior turns instead of re-sending full history