The Real Cost of RAG Pipelines in 2026

RAG is input-heavy by design

Retrieval-augmented generation pipelines stuff retrieved context into every prompt. A typical RAG call uses 6,000 input tokens and 1,500 output tokens.

That input-heavy profile makes model choice critical for total cost. Input pricing varies 20x across the registry.

Cost per call across all 14 models

Using 6,000 input and 1,500 output tokens per call:

Flagship tier:

• -Claude Fable 5: (6,000/1M x $10) + (1,500/1M x $50) = $0.135/call
• -GPT 5.6 Sol: (6,000/1M x $5) + (1,500/1M x $30) = $0.075/call
• -Claude Opus 4.8: (6,000/1M x $5) + (1,500/1M x $25) = $0.0675/call
• -GPT 5.5: (6,000/1M x $5) + (1,500/1M x $30) = $0.075/call

Mid tier:

• -Claude Sonnet 5: (6,000/1M x $3) + (1,500/1M x $15) = $0.0405/call
• -GPT 5.6 Terra: (6,000/1M x $2.50) + (1,500/1M x $15) = $0.0375/call
• -Gemini 3.1 Pro: (6,000/1M x $2) + (1,500/1M x $12) = $0.030/call

Fast tier:

• -Gemini 3.5 Flash: (6,000/1M x $1.50) + (1,500/1M x $9) = $0.0225/call
• -Claude Haiku 4.5: (6,000/1M x $1) + (1,500/1M x $5) = $0.0135/call
• -GPT 5.6 Luna: (6,000/1M x $1) + (1,500/1M x $6) = $0.015/call
• -Gemini 3 Flash: (6,000/1M x $0.50) + (1,500/1M x $3) = $0.0075/call
• -Gemini 3.5 Flash-Lite: (6,000/1M x $0.25) + (1,500/1M x $1.50) = $0.00375/call

Monthly cost at 10,000 calls per day

• -Fable 5: $0.135 x 10,000 x 30 = $40,500/month
• -Opus 4-8: $0.0675 x 10,000 x 30 = $20,250/month
• -Gemini 3.1 Pro: $0.030 x 10,000 x 30 = $9,000/month
• -Haiku 4-5: $0.0135 x 10,000 x 30 = $4,050/month
• -Gemini 3 Flash: $0.0075 x 10,000 x 30 = $2,250/month

The gap between Fable 5 and Gemini 3 Flash is $38,250/month for the same workload.

Caching changes the math for RAG

RAG pipelines often send the same system prompt and retrieved chunks repeatedly. Cache hit rates of 60-80% are common in production.

At 70% cache hit rate on input tokens with Anthropic's 90% cache discount:

• -Opus effective input rate: $5.00 x 0.30 + $5.00 x 0.10 x 0.70 = $1.85/MTok
• -Sonnet 5 effective input rate: $3.00 x 0.30 + $3.00 x 0.10 x 0.70 = $1.11/MTok

Google offers a 75% cache read discount. Anthropic offers 90% off cached reads. For high-cache workloads, Anthropic models become significantly cheaper than list price suggests.

The right model for RAG depends on QA confidence

RAG is a QA task. Here are the QA confidence scores with cost per call:

• -GPT 5.6 Terra: 92/100 QA, $0.0375/call
• -GPT 5.4: 91/100 QA, $0.0375/call
• -GPT 5.6 Sol: 91/100 QA, $0.075/call
• -GPT 5.5: 90/100 QA, $0.075/call
• -Claude Sonnet 5: 89/100 QA, $0.0405/call

GPT 5.6 Terra offers the best QA confidence per dollar in the mid tier.

Run your RAG cost numbers at /teardown, or track your pipeline spend with a free dashboard.