The Real Cost of RAG Pipelines in 2026

Updated 2026-08-16: two corrections. Gemini 3.5 Flash-Lite is priced here at $0.25/$1.50, which Google never charged - the live price is $0.30/$2.50 - and the caching section claimed Google discounts cached reads 75%. Google discounts them 90%, the same as Anthropic and OpenAI, which removes the Anthropic cache advantage the section concluded with. Both are corrected below.

RAG is input-heavy by design

Retrieval-augmented generation pipelines stuff retrieved context into every prompt. A typical RAG call uses 6,000 input tokens and 1,500 output tokens.

That input-heavy profile makes model choice critical for total cost. Input pricing varies 20x across the registry.

Cost per call across all 14 models

Using 6,000 input and 1,500 output tokens per call:

Flagship tier:

• -Claude Fable 5: (6,000/1M x $10) + (1,500/1M x $50) = $0.135/call
• -GPT 5.6 Sol: (6,000/1M x $5) + (1,500/1M x $30) = $0.075/call
• -Claude Opus 4.8: (6,000/1M x $5) + (1,500/1M x $25) = $0.0675/call
• -GPT 5.5: (6,000/1M x $5) + (1,500/1M x $30) = $0.075/call

Mid tier:

• -Claude Sonnet 5: (6,000/1M x $3) + (1,500/1M x $15) = $0.0405/call
• -GPT 5.6 Terra: (6,000/1M x $2.50) + (1,500/1M x $15) = $0.0375/call
• -Gemini 3.1 Pro: (6,000/1M x $2) + (1,500/1M x $12) = $0.030/call

Fast tier:

• -Gemini 3.5 Flash: (6,000/1M x $1.50) + (1,500/1M x $9) = $0.0225/call
• -Claude Haiku 4.5: (6,000/1M x $1) + (1,500/1M x $5) = $0.0135/call
• -GPT 5.6 Luna: (6,000/1M x $1) + (1,500/1M x $6) = $0.015/call
• -Gemini 3 Flash: (6,000/1M x $0.50) + (1,500/1M x $3) = $0.0075/call
• -Gemini 3.5 Flash-Lite: (6,000/1M x $0.30) + (1,500/1M x $2.50) = $0.00555/call

Monthly cost at 10,000 calls per day

• -Fable 5: $0.135 x 10,000 x 30 = $40,500/month
• -Opus 4-8: $0.0675 x 10,000 x 30 = $20,250/month
• -Gemini 3.1 Pro: $0.030 x 10,000 x 30 = $9,000/month
• -Haiku 4-5: $0.0135 x 10,000 x 30 = $4,050/month
• -Gemini 3 Flash: $0.0075 x 10,000 x 30 = $2,250/month

The gap between Fable 5 and Gemini 3 Flash is $38,250/month for the same workload.

Caching changes the math for RAG

RAG pipelines often send the same system prompt and retrieved chunks repeatedly. Cache hit rates of 60-80% are common in production.

At 70% cache hit rate on input tokens with Anthropic's 90% cache discount:

• -Opus effective input rate: $5.00 x 0.30 + $5.00 x 0.10 x 0.70 = $1.85/MTok
• -Sonnet 5 effective input rate: $3.00 x 0.30 + $3.00 x 0.10 x 0.70 = $1.11/MTok

Google discounts cached reads by the same 90%, so the arithmetic above applies unchanged to the Gemini rows - caching does not hand Anthropic an edge here. What does differ is the write side: Anthropic charges a 25% premium to write a prefix into cache and Google charges nothing, which matters for RAG specifically, because retrieved chunks change on most calls.

The right model for RAG depends on QA confidence

RAG is a QA task. Here are the QA confidence scores with cost per call:

• -GPT 5.6 Terra: 92/100 QA, $0.0375/call
• -GPT 5.4: 91/100 QA, $0.0375/call
• -GPT 5.6 Sol: 91/100 QA, $0.075/call
• -GPT 5.5: 90/100 QA, $0.075/call
• -Claude Sonnet 5: 89/100 QA, $0.0405/call

GPT 5.6 Terra has the best QA confidence per dollar in the mid tier.

Run your RAG cost numbers at /teardown, or track your pipeline spend with a free dashboard.