How does prompt caching cut cost?

Many applications send the same large block of text over and over: a long system prompt, a set of instructions, a reference document. Reprocessing that block on every request is wasted work, and you pay input rates for it each time. Prompt caching fixes that by storing the result of processing the unchanging front of a prompt, so later requests reuse it.

Caching works on a prefix match: the cached part has to be the exact same bytes from the very start of the prompt. Anthropic’s docs note that a cache hit “require[s] 100% identical prompt segments” up to the cached point.^[1] One change near the beginning — a timestamp, a reordered line — invalidates everything after it. So the design rule is to put stable content first and volatile content (the actual question, per-request IDs) last.

A coding assistant makes the rule concrete: a large system prompt plus the relevant slice of a codebase sits at the front of every request and barely changes call to call, while the specific question about a specific file changes every time. Put the system prompt and codebase context first so they can be cached, and the per-request question last, since that part is guaranteed to miss anyway.

The savings are large. A cache read is billed at a small fraction of the base input price — Anthropic prices reads at 0.1 times the normal input rate, with a one-time write that costs 1.25 times the base price for a 5-minute cache or 2 times for a 1-hour cache.^[1] OpenAI’s caching is automatic and kicks in for prompts of 1,024 tokens or longer. Cached tokens are billed at 10% of the standard input rate across its GPT-5.6 tiers — a flat 90% discount.^[3] How long a cached prefix stays warm depends on the model and retention policy: newer models guarantee at least 30 minutes and can hold it longer, while older models typically stay warm for 5 to 10 minutes of inactivity — up to an hour under standard retention, or up to 24 hours under an extended-retention policy.^[2] Either way, caching pays off most when the same prefix is reused before the entry falls out.