Decomposition

Lesson 1 ended with a category you can only reach after ruling out the other two: the task really is too hard as posed. This is the main tool for that case, and it is unglamorous. You stop trying to write a better prompt and start writing fewer things per prompt.

1One prompt, five jobs, one number

A prompt doing five things fails at the hardest one and hands you a single output. That output is your only signal, and it is the wrong shape for debugging. With five jobs fused into one call, you cannot:

• tell which step was wrong when the answer is wrong,
• change one step without risking the other four,
• test a step against its own examples,
• put the easy steps on a cheaper model, or in plain code,
• log an intermediate that anyone could review.

There is an arithmetic version of the same problem. Five steps at 95% each is about 77% end to end, if the errors are independent. Be careful with that number — real errors are correlated, so it is a direction, not a measurement. What is reliable is the shape of it: accuracy multiplies down, and a fused prompt hides which factor is dragging.

There is an attention version too. Your instructions for step 5 sit a long way from the data step 1 needs, and every sub-task competes with the others for the model's limited willingness to follow instructions. The hardest job loses that competition, every time.

2The chain: extract, then classify, then write

The shape that works most often is boring: pull out facts, decide something, produce prose. Each step has a narrower job and an output you can check.

Look at step 2 for a second. Once it is on its own, with a JSON input and a four-value output, an obvious question appears: does this need a model at all? Refund policy is usually a handful of rules about dates and amounts, and rules are cheaper, faster and exactly repeatable in code. You could not see that while it was buried in the middle of a paragraph. Splitting shows you which parts were never LLM work.

import json
from anthropic import Anthropic

client = Anthropic()

def ask(system: str, user: str, max_tokens: int) -> str:
    resp = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=max_tokens,
        temperature=0,
        system=system,
        messages=[{"role": "user", "content": user}],
    )
    return resp.content[0].text.strip()

EXTRACT = """Extract from the email. Return one JSON object and nothing else.
Keys in this order: order_id, purchase_date (YYYY-MM-DD), product, problem.
Use null for anything not stated. Never guess a value."""

DECIDE = """You apply refund policy. The input is a JSON object.
Reply with exactly one word: refund, replace, decline, needs_human.
Use needs_human whenever order_id or purchase_date is null."""

email = "Hi - the lamp I bought in March arrived cracked. Order 8812."

facts = json.loads(ask(EXTRACT, email, 300))
print(facts)                                  # this line is the entire point of splitting
decision = ask(DECIDE, json.dumps(facts), 5)
print(decision)

That print(facts) is not debugging clutter. It is the thing you bought. You can log it, assert on it, alert when order_id is null more than usual, and hand it to someone who does not read Python. None of that was possible when it lived inside one model's head.

Put the seams where you would want to look at the intermediate result. A seam you would never inspect costs latency and buys nothing.

3What it costs, and when one prompt wins

The last one is the one people underestimate. The customer wrote "third time this month" and your extractor did not have a field for it, so the reply is polite and clueless. Every seam is a lossy handoff. When you design the intermediate shape, you are deciding what the rest of the pipeline is allowed to know.

The last cell is not a joke. Decomposition is a debugging tool, not a virtue. A single prompt that passes its tests does not need splitting because it looks monolithic.

Routing is decomposition's cousin, and often the better first cut. Instead of one prompt with five conditional branches inside it, make one cheap classification call, then dispatch to whichever specialised prompt matches. A tiny max_tokens=5 classifier costs almost nothing, and every downstream prompt stays short, focused and independently testable. Adding a sixth category means writing a sixth prompt, not editing the one prompt that all five other categories depend on.

Two things to get right when routing: give the classifier an explicit fallback category — other, going to a generic handler or a human — so it never has to force a bad fit; and log the routing decision, because "which branch did this take" is the first question you will ask when something goes wrong.

Next: the technique that makes a single hard step work better, and the two-thirds of tasks where it is pure waste.