The system prompt and the user turn

The Messages API gives you a system parameter sitting beside the messages list, and it is easy to read that as tidiness — a place to put the boilerplate. It is more than that. Moving a line across that boundary changes how much weight it carries, whether it survives a long conversation, and what you pay for it.

1Two slots doing two different jobs

Mechanically, everything ends up as one flat sequence of tokens. build-with-llms lesson 1 makes that point and it stays true here. But "it is all tokens in the end" does not mean the slots are interchangeable, for three reasons.

Position. The system prompt is at the very front of the context, on every single call, ahead of the whole conversation. As a chat grows, a rule you typed into turn 2 slides further and further back behind newer text. A rule in the system prompt never moves.

Training. Models are post-trained on data where the system prompt is the operator's standing instruction and the user turn is the request being served. That is a learned prior, and it is a strong one: when the two conflict, the system prompt usually wins.

Stability. The system prompt is the same bytes on every call, which is exactly what prompt caching needs.

Say the honest version of the second point out loud, because it gets oversold: usually wins is not always wins. There is no enforcement in the API. A user message containing convincing instructions can still steer the model. Treat the system prompt as a reliable default, never as a security boundary — if a wrong answer would be expensive, check the answer in your code.

The practical split is about what changes:

There is a second reason to keep rules out of the user turn, separate from caching. A rule that arrives in a user message reads as this request's preference. The next user message can talk it out of that. A rule in the system prompt reads as policy, and policy is harder to argue with.

Anything you would be annoyed to see change between calls belongs in the system prompt.

2The stable half is the half you stop paying for

Prompt caching lets the provider reuse the work it already did on a prefix of your request. The match is on an exact prefix — same tokens, same order, from the very start. That single detail decides your whole layout.

Put 2,000 tokens of rules and examples at the front, unchanged on every call, and they are cacheable. Put the user's document in front of them and nothing is: the prefix is different every time, so the reusable part starts and ends nowhere.

from anthropic import Anthropic

client = Anthropic()

SYSTEM = """You write release notes for a developer tool.
One line per user-visible change. Plain past tense. No bullets, no headings.
If a change has no user-visible effect, omit it entirely."""

resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=500,
    temperature=0,
    # A list of blocks, not a plain string, so the stable part can be cached.
    system=[{
        "type": "text",
        "text": SYSTEM,
        "cache_control": {"type": "ephemeral"},
    }],
    messages=[{"role": "user", "content": "fix(auth): correct token expiry check"}],
)

print(resp.content[0].text)
print(resp.usage)  # cache_creation_input_tokens, then cache_read_input_tokens

Two things worth knowing before you rely on this. There is a minimum length below which a prefix will not be cached, and it differs by model — check the caching docs rather than trusting a number you read somewhere. And the cache entry expires, so a low-traffic endpoint may miss most of the time. Read usage on real traffic instead of assuming; the two counters above tell you exactly what happened.

3Multi-turn: the history argues back

The API is stateless. Your code resends the entire conversation on every turn, so the model re-reads all of it, every time. That has a consequence people find genuinely surprising: your latest instruction is competing with everything said earlier, including things the model itself said.

Its own past replies are the strongest signal in the box. They are few-shot examples, whether you meant them that way or not. Nine turns of bulleted answers teach it that this conversation produces bullets, and one sentence at turn 6 asking it to stop is outnumbered.

The fix in every row is the same underlying move, and it is the one thing chat interfaces train you out of: the messages list is an array you construct, not a log you append to. You are free to delete a superseded turn, rewrite a bad assistant reply into the answer you wanted, collapse ten turns into one summary message, or throw the history away and send the current state as a single fresh turn. None of that is cheating. It is the only place the conversation exists.

One caution: editing history invalidates the cache from the edit point onward, and it can confuse the user if the visible transcript and the sent transcript disagree. Keep the two in sync, or keep the differences deliberate and small.

Next: what to do when describing the output still is not landing — showing it instead.