Trimming a prompt that grew

This is not a lesson about short prompts for their own sake, and it is not a way to save tokens. A short prompt that fails your cases is worse than a long one that passes. The harness from Versioning and testing prompts can tell you whether an edit helped. It can just as easily tell you whether a line still earns its place.

1.Prompts only grow, and some lines go stale

Every failure adds a line. Nothing removes one. The evidence is lopsided: a rule you add has a failing ticket behind it, so it feels earned, while a rule you could delete has nothing behind it. No case fails because a line is unnecessary. After a year the prompt is sediment: fixes for failures the current model no longer has, in a style an older model needed.

One team's report of cutting its agent's system prompt hard and keeping one prompt per model is told in The system prompt and the user turn; it is an anecdote, so take the habit and not the number: a prompt belongs to a model, so re-check it when the model changes.

Which lines are the suspects? The vendor's own guides name several, as of October 2026.

One more suspect needs no model name. A rule that is true nine times in ten misfires on the tenth. The Claude Code team's "always verify the front end" rule, rewritten to say when to run the app, is the case told in Why prompts fail. Ask of every rule, "when is this wrong?" If you can name the case, write the edge into the rule. If you cannot, it is a candidate to cut. Examples beat explanations has the same dispute about examples, with the same answer: test it.

A line stays in the prompt because a case needs it, not because someone once added it.

2.Remove one block at a time

The method is ablation, and it is the harness from Versioning and testing prompts used backwards.

1. Split the prompt into blocks that each do one job. Leave the core (task, labels, reply format) alone; you are testing what grew around it.
2. Pin the model, effort and thinking mode, then run the full prompt twice. The gap between the runs is your noise, and it sets how big a move you may believe.
3. Remove one block, run the same cases with the same repeats, and read the share right every time. Note which block went and the score, so the run can be replayed.
4. Keep a deletion when the score stays inside the noise. Restore any block whose removal moves it further.
5. Put the survivors together and run once more. Two blocks can each look free alone and still be needed together, because each covers for the other.

Score the share right every time, not the average: a deletion can leave the average alone while making single cases flaky. A June 2025 Wharton study, on older models and one science benchmark, found questions that were not answered the same way every time even at temperature 0. So at twenty cases, one case moving is probably wobble. Read that case before you trust the count.

Here is that lesson's classifier a year on: eight lines of core and six blocks around them, forty in all.

Cutting everything that did not move leaves twelve lines. Run that prompt once more as a whole. If it lands inside the noise of 16 or 17, you have a prompt a third the size that measures the same on your cases. If not, restore blocks one at a time until it does.

The bill is small for a one-word label: eight runs of twenty cases at five repeats is at most 800 short calls, and a variant would pay its own cache write if you cache the prefix and it clears the model's minimum length (Prompt layout and the cache). It also shows what trimming is for: a stable prefix is cached, so the lines you cut were cheap to carry (as of October 2026). The gain is behaviour and clarity.

3.The script, a rule it deleted, and when to leave it alone

The prompt file from Versioning and testing prompts now holds named blocks, and SYSTEM is still their join, so that lesson's run script keeps working. The parameter spellings below are copied from vendor docs as of October 2026, not run here, so try them on your SDK version first.

# prompts/classify_ticket.py — the same file, with the prompt held as named blocks
MODEL = "claude-sonnet-5-5"   # as of October 2026
EFFORT = "high"
THINKING = {"type": "between_tools"}   # thinking off: a label needs none
BLOCKS = {   # one job each, in prompt order; shortened here, the real ones run to the table's line counts
    "core": "Classify the ticket: billing, bug, feature_request or other. Reply with the label only.",
    "emphasis": "CRITICAL: NEVER guess. ALWAYS answer.",
    "steps": "First read the ticket, then list its problems, then pick.",
    "examples": open("prompts/examples.txt").read(),   # twelve lines: three worked tickets
    "reminders": "Remember: the label only. No punctuation.",
    "billing_or_bug": "A charge the customer says is wrong is billing, even if they call it a bug.",
    "forwarded_thread": "If the ticket quotes an earlier email thread, label the newest message only.",
}
SYSTEM = "\n\n".join(BLOCKS.values())   # evals/run.py imports this, unchanged

# evals/ablate.py
import json

from anthropic import Anthropic

from prompts.classify_ticket import BLOCKS, EFFORT, MODEL, THINKING

client = Anthropic()
REPEATS = 5

def text_of(resp) -> str:
    return "".join(b.text for b in resp.content if b.type == "text")

def run_case(system: str, case: dict) -> str:   # the same call as before, with the prompt passed in
    resp = client.messages.create(
        model=MODEL, max_tokens=20, thinking=THINKING, output_config={"effort": EFFORT},
        system=system, messages=[{"role": "user", "content": case["input"]}],
    )
    if resp.stop_reason != "end_turn":
        return f"[{resp.stop_reason}]"   # the case fails and the run goes on
    return text_of(resp).strip().lower()

def right_every_time(blocks: dict, cases: list[dict]) -> int:
    system = "\n\n".join(blocks.values())
    return sum(all(run_case(system, c) == c["expected"] for _ in range(REPEATS)) for c in cases)

cases = json.load(open("evals/tickets.json"))
print("full prompt:", right_every_time(BLOCKS, cases), "and again:", right_every_time(BLOCKS, cases))
for name in BLOCKS:
    if name != "core":   # the task itself is not on trial
        rest = {k: v for k, v in BLOCKS.items() if k != name}
        print(f"without {name}:", right_every_time(rest, cases))

A failure the harness missed. In the table, the forwarded-thread rule scored the same without it, so it went. It scored the same because none of the twenty cases was a forwarded thread. Picture a few weeks later: customers forward long threads with a new bug report on top, and the quoted refund talk underneath pulls tickets into billing. The twenty never asked. The fix is one case per rule before you delete anything. For each block, find the real ticket that made someone add it, or write one, and run it without the line. A rule whose case still passes is cleared. A rule with no case is not cleared, only unmeasured: leave it, or write the case first. List what you removed in the commit message, so the next regression can find the old text.

When not to use it. Do not trim before the harness exists; deleting by taste is just editing by feel. Do not carry a trimmed prompt to another model. A smaller or older model may still need the long version, and the team said it has not tested whether a big model's prompt lifts a small one. The LLMs course has the migration recipe in Switching models safely. Never delete a format or safety rule that only the prompt enforces; move it into code first, as a schema (Structured outputs in practice), a validator (Validating what comes back) or a gate on untrusted text (Containing prompt injection). And skip all this for a prompt that is already ten lines.

Be honest about how little this proves. The case for leaner prompts here is one team's report, plus a claim in the same post that another vendor's guide agrees, which was not checked here. You are getting a method, not a target.

Next: trimming is a search with one move, which is to delete a block. Let a model write the prompt gives the search more moves, and lets a model propose them.