Prompt layout and the cache

This is not a lesson about saving money, though it will. It is about the order of a request, and why one order suits the model and the cache at once. The system prompt and the user turn decided which slot a line belongs in. This one decides where it sits inside the slot.

1.The request has an order, and the cache keeps a prefix

The API reads a request in a fixed order: tool definitions, then the system prompt, then the messages. Prompt caching stores the work done on a prefix of that sequence, up to a block you mark with cache_control. A later request that starts with identical content up to that block reads the stored work. From the first changed token onward, everything is processed fresh. The vendor says the output is the same either way (as of October 2026).

That gives one layout rule.

Stable first, volatile last.

Tools, standing rules, the style guide and the examples go at the front. The document goes next, then the date, then the question.

The same order helps the model on long inputs. The vendor's advice (as of October 2026) is that past about 20,000 tokens you put the documents above the question, and ask the model to pull out the relevant quotes before it answers. The vendor claims the question at the end can lift quality by up to 30 percent on complex multi-document inputs. That figure is vendor-run, with no test details published, so try it on your own cases. Wrap each document in tags, <documents> around <document index="1"> with <source> and <document_content> inside, and keep the tag names the same across calls. Tags help the model tell the pieces apart. Quote-first looks like one sentence at the end of the user turn: copy the passages that bear on the question into <quotes> tags, then answer from them. It costs a few output tokens and gives you something to check the answer against. The tags are not a security boundary, as Prompt injection explains.

any tool definitions1
the style guide and twelve example commits2
the date, the diff, the request3

Two ways to mark the breakpoint. Automatic mode is one top-level switch, and the breakpoint moves forward each turn, which suits chat. Explicit mode puts cache_control on a block, up to four of them, for parts that change at different rates. Either way, the entry lasts five minutes and each read refreshes it; an optional "ttl": "1h" suits slower traffic. There is a minimum cacheable length that depends on the model: 512 tokens on Sonnet 5.5 and several other current models, 1,024 to 4,096 on older ones, as of October 2026. Below it nothing is cached and nothing errors. Reads cost a fraction of normal input price; the LLMs course has the table in What you actually pay for. Since a write costs a little more than plain input and a read saves most of it, one reuse inside the window probably pays back the write. That is arithmetic on the vendor's multipliers, not a vendor claim.

2.A worked example: release notes with a big style guide

Back to the release-notes generator. The system prompt holds a style guide and twelve example commits, about 3,000 tokens that never change. Each call brings a diff and today's date.

from datetime import date

from anthropic import Anthropic

client = Anthropic()
MODEL = "claude-sonnet-5-5"   # as of October 2026

def text_of(resp) -> str:
    return "".join(b.text for b in resp.content if b.type == "text")

# About 3,000 tokens, well over the 512-token minimum, identical on every call.
STYLE_GUIDE = open("release_notes_style.md").read()

def write_notes(diff: str) -> str:
    resp = client.messages.create(
        model=MODEL,
        max_tokens=2000,   # thinking is on by default and counts against this
        system=[{
            "type": "text",
            "text": STYLE_GUIDE,
            "cache_control": {"type": "ephemeral"},   # the last block that never changes
        }],
        messages=[{
            "role": "user",
            # The date and the diff come after the breakpoint, in the user turn.
            "content": f"<today>{date.today()}</today>\n<diff>\n{diff}\n</diff>\n\nWrite the release notes.",
        }],
    )
    u = resp.usage
    print(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens)
    if resp.stop_reason != "end_turn":
        raise RuntimeError(f"no usable answer: {resp.stop_reason}")
    return text_of(resp)

The breakpoint sits on the style guide because that is the last block identical on every call. The date and diff land after it, so they are charged as ordinary input and never poison the stored part. The parameter spellings are copied from the vendor docs as of October 2026, not run here, so try them on your SDK version first.

Total input is the sum of the three counters. input_tokens is only the part after the last breakpoint, so a small value on a hit is the success case, not a bug. It also means every prompt edit costs one write. Expect that after each change to the guide.

3.What breaks it silently, and when to skip it

Someone wants the model to know the date, so the first line of system becomes Today is 2026-10-17. A plain date changes once a day, so it costs one fresh write a day, not a miss on every call. The real breaker is a timestamp to the second, which costs one write per call. The first line is then different on every call, so no stored prefix ever matches. Every call pays the write price, none reads, and cache_creation_input_tokens is full every time. The output is fine and there is no error. You find out from the invoice. The fix is the sample's: move the date into the user turn, after the breakpoint. The vendor says automatic mode has the same trap, because it marks the last block, so a last block holding a timestamp or the user's question stores nothing stable.

Other quiet breakers, from the vendor's troubleshooting list (as of October 2026): changing tool_choice, adding or removing an image, or changing the thinking or effort settings partway through a cached conversation. Editing a tool definition is worse, because tools come first. Editing or reordering earlier turns moves the point where the prefix stops matching, so keep history append-only. In some languages, randomised key order in tool inputs can do it too. The vendor also tabulates which setting invalidates which level; read it on the vendor page.

To see why a call missed, the vendor's cache diagnostics (generally available, Claude API only, as of October 2026) takes the id of your previous response and says where the prefix first diverged: system_changed, tools_changed, messages_changed and so on. A *_changed reason with low reads is your bug. No reason with low reads usually means the entry expired, so shorten the gaps or use 1h.

Be honest about the shelf life here. Minimums, lifetimes and multipliers have all moved more than once, which is why each number above is dated. The layout rule has not moved. Read the vendor page before you size anything by a figure in this lesson.

When not to bother. A prompt under the model's minimum has nothing to gain. Traffic with gaps over five minutes and no 1h option writes every time, at a premium. Do not bend a prompt's clarity to chase the cache: if the question at the end makes a short prompt read worse, test it both ways. And the cache is not a privacy control. It is the same text, held for a while, and it makes nothing safer to put in a prompt.

Next: a stable prefix is the best place for examples, and Examples beat explanations is about choosing them.