Structured output

This is the lesson that turns a demo into software. A model that returns a paragraph is a toy you read. A model that returns a validated object is a function you can call from the rest of your system.

1You cannot regex your way out of free text

The first attempt always looks the same. Ask for JSON in the prompt, then clean up whatever comes back.

It works, then it does not:

• Sometimes the object arrives naked. Sometimes wrapped in a fenced code block. Sometimes with Here is the JSON you requested: in front.
• A long value contains a } and your brace-matching falls over.
• One run emits "total": "1,299.00", the next emits "total": 1299.0.
• The model decides a field is worth explaining and adds a key you never asked for.
• On a truncated response you get half an object, which parses as nothing.

Each fix is one line. Ten fixes later you own a parser with no test suite, failing in production on inputs you cannot reproduce.

The reason this is unwinnable goes back to lesson 1. The model is picking likely next tokens. After a request for structured data, a friendly preamble is still a very likely continuation, because that is how helpful text usually reads. Your prompt asked. Nothing stopped it. Politeness is not a constraint.

A prompt asks for a shape. A schema makes the shape the only thing the model is allowed to produce.

2Borrow the tool-calling machinery for its guarantee

Tool use exists so a model can call your functions — that is lesson 5. But it comes with a property you want right now: when the model calls a tool, the arguments must match the JSON Schema you declared for that tool. That constraint is enforced during decoding, not requested in prose.

So define a tool that does not do anything. It exists only to have a shape.

Then force it. Anthropic's API takes tool_choice={"type": "tool", "name": "..."}, which means the reply must be a call to that tool. No prose path is available. You read tool_use.input and you have a dict.

Writing the schema well is where the accuracy lives, and most of it is unglamorous:

• Enums, everywhere the answer is a finite set. "status": {"type": "string", "enum": ["paid", "unpaid", "partial"]} removes Paid, PAID, paid in full and unpaid (overdue) in one line. This is the cheapest accuracy win in the whole course.
• required for the fields you truly need, and a nullable type for the ones that may genuinely be absent. Forcing a field to exist when it might not is how you invite an invented value.
• A description on every field. The model reads them. "issue_date" with the description The date on the invoice itself, not the date it was received. ISO 8601, YYYY-MM-DD. fixes a whole class of confusion that no amount of prompt text will.
• Types that mean something. Numbers as number, not string. Dates as a string with the format spelled out in the description. Booleans as boolean.
• Flat beats nested. Deep object trees are harder for the model and harder for you to validate. Flatten unless the nesting is real.

3Validate anyway, and retry with the error

The schema guarantees the shape. It does not guarantee the truth. issue_date will be a string; it may still be a date that appears nowhere in the document. So the object crosses one more gate before it enters your system.

Define the same shape a second time in your own language — Pydantic in Python, Zod in TypeScript — and parse the model's output through it. That gives you three things the schema alone does not: real types on your side of the wire, range and format checks the JSON Schema cannot express, and a precise error message when something is off.

That error message is the point. When validation fails, do not retry the identical request — you will draw another sample from the same distribution, and the odds of the same failure are high. Instead send the failed object and the validation error back as another turn, and ask for a corrected object. The model is good at this, because now it is fixing a specific named problem rather than guessing again.

A retry policy that holds up:

• Attempt 1 fails validation → send back the error text, ask for a fix.
• Attempt 2 fails again → stop. Two failures usually means the input is not what you think it is, not that the model is having a bad day.
• On giving up, log the raw response and the input, and fail loudly. A silent default written into a required field is a bug that surfaces three weeks later in a report.

Cap the retries. An unbounded fix-it loop is an unbounded bill.

from anthropic import Anthropic
from pydantic import BaseModel, ValidationError
from typing import Literal

client = Anthropic()

class Invoice(BaseModel):                    # your side of the contract
    invoice_number: str
    total_amount: float
    currency: Literal["USD", "EUR", "GBP"]
    status: Literal["paid", "unpaid", "partial"]

# A tool that is never executed. It exists purely to pin the output shape.
EXTRACT_TOOL = {
    "name": "record_invoice",
    "description": "Record the fields extracted from an invoice.",
    "input_schema": {
        "type": "object",
        "properties": {
            "invoice_number": {"type": "string", "description": "The invoice's own reference number."},
            "total_amount": {"type": "number", "description": "Grand total including tax."},
            "currency": {"type": "string", "enum": ["USD", "EUR", "GBP"]},
            "status": {"type": "string", "enum": ["paid", "unpaid", "partial"]},
        },
        "required": ["invoice_number", "total_amount", "currency", "status"],
    },
}

def extract(document: str) -> Invoice:
    messages = [{"role": "user", "content": document}]
    for attempt in range(2):                 # one try, one repair, then stop
        resp = client.messages.create(
            model="claude-sonnet-5",
            max_tokens=1000,
            temperature=0,
            tools=[EXTRACT_TOOL],
            tool_choice={"type": "tool", "name": "record_invoice"},  # prose is not an option
            messages=messages,
        )
        block = next(b for b in resp.content if b.type == "tool_use")
        try:
            return Invoice(**block.input)    # shape guaranteed, values still checked
        except ValidationError as e:
            messages += [
                {"role": "assistant", "content": resp.content},
                {"role": "user", "content": f"That failed validation:\n{e}\nCall the tool again, corrected."},
            ]
    raise RuntimeError("invoice extraction failed twice")

Notice what is missing from that file: no regex, no json.loads in a try, no stripping of code fences. The parsing problem did not get solved. It got deleted.