What the loop remembers within one run

1Why the transcript forgets

The transcript is everything you send on every step: the system prompt, the tool definitions, the task, and then every tool call and every tool result the run has produced so far. It only grows. Three things go wrong as it does, and they go wrong in this order.

Signal thins out. Your constraint is one sentence. Between it and the current step sit thirty tool results, several of which are four hundred lines of JSON. The sentence is still in the window and the model still misses it. This is the same effect the LLMs course covers in Context windows and their limits, arriving from a direction that lesson does not: nobody wrote the noise, a loop generated it.

You pay for all of it, every step. Step forty re-sends steps one to thirty-nine. That is why cost inside a loop is quadratic-ish in the number of steps rather than linear, and why a run that is comfortably inside its step budget can still be far outside anything you would have signed off.

Then it fills. You hit the window and the request fails, or your harness truncates from the front and quietly deletes the oldest thing in the conversation - which, since the task and its constraints were the first thing you sent, is the task and its constraints.

Not everything in there is equally worth keeping:

That last row is most of your context and nearly all of your growth. It is also the least valuable per token after the step that used it.

2State that lives outside the window

The fix costs almost nothing and it is a change of habit rather than of stack: stop using the conversation as storage.

Three shapes, and they compose.

A scratchpad the agent writes to. Give it a tool that writes a note to a file, and one that reads it back. Anthropic calls this structured note-taking, and the reason it works is dull: a note on disk is not affected by anything that happens to the transcript. The agent records what it found, the transcript can be trimmed to nothing, and the finding is still there.

A state block your harness re-injects. Your code owns the invariants - the task, the constraints, what has been done, the current sub-goal, the budget spent - renders them, and appends them at the end of every step. The model cannot lose what you keep handing back. This is the direct fix for the opener, it is about twenty lines, and it is the one to write first.

A task list. The same idea with a second benefit: it gives you a progress signal you can read without a model call, which is what Planning a run, and knowing when to stop's no-progress detector wants.

The last row is the honest one. Both alternatives have a failure mode, and both are better failure modes than the transcript's, because both are things your code can see and fix.

Anything the run must not forget should not live in the transcript alone. Keep it where your code can put it back.

3Compaction, and what it actually costs

When the window is genuinely under pressure, there are two mechanics, and they are routinely confused.

Tool-result clearing drops the oldest tool results from the history and keeps the last few. The model still knows it called search at step six; it no longer carries the four hundred lines that came back. Anthropic ships this as a context_management edit called clear_tool_uses_20250919, with a trigger threshold, a keep count, and an exclude_tools list for the ones you never want cleared. Lightest touch, most predictable loss.

Compaction summarises the run so far and continues from the summary. Anthropic's version is compact_20260112: when it fires, the response carries a compaction content block, and on later requests everything before that block is dropped. You have to pass the block back or the next request will not know where the summary starts. There is a matching clear_thinking_20251015 for thinking blocks.

Those type strings carry dates for a reason. They are beta surfaces and the names move - check the context editing and compaction docs before you copy any of them.

The costs, none of which the feature pages lead with:

• It is lossy in an unpredictable direction. If step six produced the exact error string that turns out to matter, and the summary says "the build failed", the string is gone and no future step can get it back.
• It is a model call. Latency you did not budget for, on the step where it fires, plus a chance of being wrong.
• It breaks your prompt cache. Everything before the compaction point changed, so the cached prefix is dead and the next step pays full price for the whole window.
• It is not a substitute for a scratchpad. Compact the conversation. Keep the facts somewhere a summariser cannot blur them.

import json
from anthropic import Anthropic

client = Anthropic()

# Facts the run must not lose. Your code owns these; the transcript does not.
state = {
    "task": "Reconcile the March invoices against the ledger.",
    "constraints": ["Never write to the ledger.", "Ignore invoices under 50 GBP."],
    "checked": [],
}

def user_turn(tool_results, state):
    # tool_result blocks must come first in the turn; text goes after them.
    return {
        "role": "user",
        "content": tool_results + [{
            "type": "text",
            "text": ("Run state, authoritative - prefer this over anything earlier "
                     "in the conversation:\n" + json.dumps(state, indent=2)),
        }],
    }

def step(messages, tools):
    resp = client.beta.messages.create(
        model="claude-sonnet-5",
        max_tokens=2048,
        tools=tools,
        messages=messages,
        betas=["context-management-2025-06-27", "compact-2026-01-12"],
        context_management={"edits": [
            # Old tool results are where the tokens are. Drop those first.
            {"type": "clear_tool_uses_20250919",
             "trigger": {"type": "input_tokens", "value": 60_000},
             "keep": {"type": "tool_uses", "value": 5}},
            # Summarise only once the window is genuinely under pressure.
            {"type": "compact_20260112",
             "trigger": {"type": "input_tokens", "value": 150_000}},
        ]},
    )
    # Append the whole response, compaction block included, or the next request
    # will not know where the summary starts.
    return messages + [{"role": "assistant", "content": resp.content}]

The two thresholds are placeholders. Pick yours from your own runs: log input tokens per step across real tasks, see where they climb, and set the clearing trigger below the point where the window starts hurting and the compaction trigger well above it. Clearing should be doing the routine work; compaction should be rare.

All of this ends when the run does. The next lesson is about what you choose to keep afterwards, and why that is a much harder problem than this one.