Caching and batch: the two big discounts

What you actually pay for listed the line items on a bill, including cache writes, cache reads and batch. This lesson covers the two that can cut a bill by half or more, and the ways each one stops working while the app carries on as normal. Every figure below is a ratio of the normal input price with a date on it, because ratios have held steadier than prices. Dollar prices are not quoted at all. Read them off the provider's page on the day you decide.

1.A cache remembers the start of your prompt

A prompt cache stores the work a provider has already done on the first part of a request. If your next request starts with exactly the same text, that work is not repeated, so the request is cheaper and starts answering sooner. It is a prefix cache: everything up to the breakpoint has to match, character for character. That is why stable content goes first and anything that changes goes last, a related ordering rule to the one Context windows and their limits gives for where to place an instruction.

How you switch it on differs by provider. As of Oct 2026:

Whatever the provider, the sum is the same. A write costs more than plain input and a read costs far less, so a prefix pays back only if it is read again. On Anthropic, as of Oct 2026, a five-minute write is repaid by one read, and a one-hour write by two. A prefix used once is a small loss.

An entry does not last. As of Oct 2026, Anthropic's default is five minutes, refreshed free every time it is read. A one-hour lifetime is opt-in with "ttl": "1h" and costs more to write. The clock runs from the start of the request that wrote or read the entry, not from the end of its answer, so a slow response eats into the gap you have left.

There is also a floor. A prefix shorter than the model's minimum is processed without caching and no error comes back. As of Oct 2026 Anthropic's minimum runs from 512 tokens on its newest models to 4,096 on some older ones.

2.How a cache stops working without telling you

The failure that costs real money is a prefix that looks stable and is not.

A timestamp is the common case, not the only one. On Anthropic, as of Oct 2026, a change to tool definitions, tool_choice, images, thinking settings or output_config.effort also invalidates part of the cache, so a tidy-up of those can switch your savings off. Thinking and effort, model by model covers the last two. Two more traps, as of Oct 2026: Anthropic checks only about 20 positions back from a breakpoint, so a single turn that adds more than that can miss the earlier entry, and parallel requests cannot read an entry until the first response has begun, so send one first.

You find all of this by reading the response, not the invoice. The script below marks the handbook as the end of the cached part and prints both fields.

from anthropic import Anthropic

MODEL = "claude-sonnet-5-5"   # check the models page; as of Oct 2026
client = Anthropic()

# Long enough to pass the minimum cacheable size, and identical on every request.
HANDBOOK = open("handbook.md").read()

def ask(question: str) -> None:
    resp = client.messages.create(
        model=MODEL,
        max_tokens=2000,   # room for the answer and any thinking
        system=[
            {"type": "text", "text": "Answer only from the handbook below."},
            {"type": "text", "text": HANDBOOK, "cache_control": {"type": "ephemeral"}},
        ],
        messages=[{"role": "user", "content": question}],
    )
    u = resp.usage
    print(f"written={u.cache_creation_input_tokens} read={u.cache_read_input_tokens} new={u.input_tokens}")

ask("How much notice do I have to give?")
ask("How many days of annual leave do I get?")

On the first call, written should equal the size of the handbook and read should be 0. On the second, the two swap, and new is only the tokens after the breakpoint. If both stay 0, the handbook is under the minimum. If written fills on both calls, something above the breakpoint changed. This request shape builds and parses with Anthropic's Python SDK, version 1.15.0, but it has not been run against the live API, so treat the numbers above as what the documentation says you will see.

Caching also changes your rate limits. Anthropic says that for most models, as of Oct 2026, cache reads do not count toward the input-tokens-per-minute limit. The field input_tokens holds only what follows the last breakpoint, so a cached 200,000-token document plus a 50-token question shows input_tokens: 50. Total input is reads plus writes plus input_tokens. That is Anthropic's own framing. How OpenAI and Google count cached tokens against their limits is not covered here. Rate limits and reliability covers the rest of the limits.

3.Batch, the arithmetic, and when neither applies

A batch is a job. You submit many requests together and collect the results later. As of Oct 2026, Anthropic, OpenAI and Google each list about 50% off, on both input and output. In return, nothing streams, and you give up the answer arriving now. As of Oct 2026, Anthropic says most batches finish within an hour but expire at 24 hours, and an expired request is unanswered and not billed. OpenAI accepts only a 24h window as of the same date. Results can come back in any order, so every request carries a custom_id and you match on that. Batches also have their own limits. As of Oct 2026 Anthropic caps one batch at 100,000 requests or 256 MB, and keeps a separate queue limit.

Caching and batch stack, but with a catch. As of Oct 2026, batch requests run concurrently and in any order, Anthropic describes cache hits inside a batch as best effort, and its page cites hit rates anywhere from 30% to 98%. It suggests the one-hour lifetime for batches.

Here is the sum for 1,000 questions over one handbook. The unit is the cost of reading the handbook once with no discount. Only the handbook is counted, not the questions or the answers.

A cache that never hits costs more than no cache. The saving is real only when the usage fields say so.

Neither discount is always right.

• A prefix used once or rarely. You pay the write premium and the entry expires before a second call. The one-hour lifetime helps only if it is read again inside the hour.
• A prefix that is too short, or that changes per user. Put the per-user text after the breakpoint, or accept that the part before it is small.
• Bursty traffic. Gaps longer than the lifetime make every burst start with a write.
• A user is waiting, you need streaming, or step two needs step one's answer now. Batch is wrong for all three.
• A hard deadline under 24 hours. An expired request comes back with nothing, and you must resubmit it live.

Batch does suit nightly enrichment, backfills and evaluation runs. The Evals course covers the habit around them in Regression testing a prompt or model change.