Prompt Caching Cost Guide: Cache Hits, Prefixes, and Real API Spend

Prompt caching cost depends on three variables: how much of your prompt is a reusable prefix, how often that prefix repeats within the cache's active window, and how a given provider prices cache writes versus cache hits. Get those three numbers right and caching can meaningfully lower your input token bill on repetitive workloads; get them wrong and you can pay a write premium for a cache that never gets reused.

This guide separates what providers document, what you can verify on public API surfaces, and what you should test before committing production spend to a caching strategy.

Key Takeaways

What Prompt Caching Actually Charges You For

Prompt caching lets an API provider store the processed representation of a prompt prefix so that subsequent requests sharing that prefix skip redundant computation. The billing model that follows from this is not "cached tokens are free." It is closer to "cached tokens are cheaper on reuse, but the first write costs more than a standard input token."

Anthropic's prompt caching documentation lays out this structure directly: a request that creates a new cache entry is billed differently from a request that hits an existing one. The exact multipliers change over time and by model, so treat any number you see in a blog post, including this one, as something to verify against the current docs rather than a fixed constant.

The practical implication is that prompt caching is a bet on reuse. If your system prompt, tool schema, or retrieved context block is sent once and never repeated, caching adds a write premium with no offsetting hit savings. If that same block is sent hundreds of times within the cache's active window, the hit savings can outweigh the write cost by a wide margin.

How Cache Hits Work: Prefixes, Prefixes, and Breakpoints

Cache hits are prefix-based, not content-based in a fuzzy sense. The cached portion of a prompt must match the incoming request token-for-token up to the point where the cache boundary, sometimes called a breakpoint, is set. Anthropic's documentation describes this as an explicit mechanism where developers mark which portion of the prompt is eligible for caching, typically the stable system instructions, tool definitions, and long reference documents that do not change between calls.

This has a direct engineering consequence: anything you place before the cache breakpoint must be byte-identical across requests, including whitespace and ordering. A common mistake is interleaving per-request variables (like a timestamp or a user ID) into the system prompt ahead of the cache boundary. That single variable defeats the cache for the entire prefix, and you pay write costs on every call instead of accruing hits.

The fix is straightforward: keep genuinely static content (tool definitions, house style instructions, large reference documents) in the cached prefix, and push anything request-specific into the uncached suffix, typically the user message.

Cache lifetime matters just as much as prefix design. Anthropic's documentation describes a default cache duration measured in minutes, with a longer-duration option available for workloads that need it. If your traffic pattern sends a shared prefix once every few minutes, a short-lived cache can expire before the next request arrives, and you end up paying write costs repeatedly. High-frequency workloads (chat sessions, agent loops, batch pipelines running back to back) are far better candidates than low-frequency, sporadic calls.

Modeling Real API Spend: A Worked Approach

Rather than asserting a savings percentage, model your own workload with the following shape. This example illustrates the request structure conceptually; check the exact field names and current pricing on the provider's documentation before you implement it.

{
  "model": "claude-sonnet-5",
  "system": [
    {
      "type": "text",
      "text": "You are a support agent. Full policy document follows...",
      "cache_control": { "type": "ephemeral" }
    }
  ],
  "messages": [
    { "role": "user", "content": "What is the refund window for order 48213?" }
  ]
}

The cache_control marker on the system block signals that this content is a caching candidate. The first call in a session pays the write cost for that block. Every subsequent call within the cache's active window that reuses the identical prefix pays the hit rate instead of the full input rate for those tokens.

To estimate whether this is worth implementing for your service, gather four numbers from your own logs:

1. Prefix size: the token count of the stable content you intend to cache (system prompt, tool schema, reference document).
2. Call frequency within the TTL window: how many requests reuse that exact prefix inside the cache's active duration.
3. Write and hit rates: pulled from current provider documentation, not assumed from memory.
4. Suffix variability: whether the non-cached portion of your prompt is small relative to the cached portion, since savings scale with how much of the total prompt sits behind the cache breakpoint.

If your prefix is large, your call frequency inside the TTL window is high, and your suffix is small, caching is likely to reduce spend. If any of those three conditions is weak, run a side-by-side cost comparison before rolling caching out broadly. TokenLab's guide on cutting AI API costs walks through a broader set of levers beyond caching, including model selection and batching, at /blog/cut-ai-api-costs-30-percent.

Decision Table: When Prompt Caching Pays Off

Workload pattern	Cache likely to help	Notes
Long system prompt or tool schema reused across many calls in a session	Yes	Classic case; write cost amortizes across hits
Large retrieved document reused across a short burst of follow-up questions	Yes, if calls land within TTL	Confirm TTL against current docs before assuming reuse window
One-off prompts with no repeat traffic	No	Write premium with no hit to offset it
High-variability prompts where the "stable" section keeps changing	No	Any change ahead of the breakpoint invalidates the cache
Agent loops with repeated tool definitions across many turns	Yes	Tool schemas are prime caching candidates
Low-frequency batch jobs spaced beyond the cache TTL	No	Cache expires before reuse; pay write cost every time
Multi-provider routing where only some backends support caching	Verify per model	Do not assume caching support transfers across providers

Use this table as a starting checklist, not a final answer. Confirm TTL, write/hit pricing, and breakpoint mechanics against the provider docs for the specific model you plan to use, since these details change and vary by model family.

Provider Differences You Should Verify Before Committing

Prompt caching is not implemented identically everywhere, and that matters if you route traffic across multiple providers or models. OpenRouter's documentation on prompt caching best practices notes that caching support and behavior differ by underlying provider, meaning a strategy tuned for one model's cache mechanics does not automatically apply when you switch models or route through a different backend.

If your architecture uses model routing to control cost (for example, sending routine classification tasks to a lower-cost model like DeepSeek V4 Flash, GLM-5.2, or Gemini 3.5 Flash while reserving Claude Sonnet 5 or GPT-5.5 for harder reasoning tasks), you need to check caching support independently for each model in that routing table. A caching strategy validated against one model's docs is not a safe assumption for another. TokenLab's rankings page tracks model-level differences you can use as a starting reference point at /models/rankings, and the routing benchmark analysis at /blog/ai-model-routing-benchmark-cost-per-task covers how routing decisions interact with per-task cost, which compounds with caching decisions rather than replacing them.

Limitations of This Analysis

This guide describes the general mechanics of prompt caching as documented by Anthropic and referenced by OpenRouter as of the observed dates above. It does not include exact write/hit multipliers, exact TTL durations, or per-model pricing, because those figures change and differ by model. Before you build a cost model for production traffic, pull current numbers directly from the provider documentation linked above rather than relying on any fixed figure quoted in third-party content, including this article. Caching behavior for reasoning-oriented models, multimodal prompts, and very long context windows may also differ from the general prefix-caching pattern described here; verify against documentation for the specific model you plan to use.

FAQ

Does prompt caching always reduce API spend? No. It reduces spend only when a stable prefix is reused frequently enough within the cache's active window to offset the write cost. Sporadic or highly variable prompts often cost more with caching enabled than without it.

What breaks a cache hit? Any change to the prompt content ahead of the cache breakpoint, including whitespace, token order, or a single variable inserted into an otherwise static system prompt. The match must be exact up to the breakpoint.

Is prompt caching implemented the same way across all providers? No. Anthropic documents an explicit cache-control mechanism with defined write and hit pricing. OpenRouter's documentation notes that caching support and pricing vary by the underlying provider and model, so you should verify support per model rather than assuming it transfers.

If you are evaluating whether prompt caching, model routing, or a combination of both fits your traffic pattern, get started with TokenLab to compare model options and cost structure before you commit production spend.