Output contracts

build-with-llms lesson 3 covers the machinery for getting a guaranteed object back: define a tool schema, force the tool choice, validate what arrives. That is the how. This lesson is the design question that comes first — what shape should the contract actually be? Most of the reliability people chase with prompt wording is available for free in the schema instead.

1The ladder from asking to guaranteeing

There are four rungs between "I would like JSON" and "this response cannot be the wrong shape".

Most people stop at rung 1, get burned, and write a parser. Rung 2 is worth knowing about because it costs almost nothing and fixes most of the drift on a small task. Rung 4 is the one that deletes the parser.

The caveat that matters more than any of this: a guaranteed shape is not a guaranteed answer. The schema says invoice_total is an integer. It does not say it is the integer on the invoice. Structure and truth are separate problems, and the schema only solves the first.

2Closed sets are the cheapest accuracy you can buy

Every free-text field is an invitation to a synonym. Declare status as a string and you will collect paid, Paid, PAID, settled, payment received and Payment Received — six spellings of one fact, and your if status == "paid" is wrong five times out of six.

An enum of five values collapses all of that into one line. It also does something less obvious and more valuable: it forces you to decide what the categories actually are. That is usually where the real ambiguity was hiding, and the model was absorbing it silently.

The confidence row deserves its own note, because that field is on almost every first draft. Asking a model to score its own confidence feels rigorous and mostly is not — the number is generated the same way every other token is, and it is not calibrated against anything. A quote you can search the source document for is worth far more, because your code can verify it and a self-reported 0.87 can only be believed.

3Name the fields, fix the order, define unknown

Three details carry most of the remaining weight.

Name them precisely. issue_date and due_date, not date. The field name is an instruction, and it sits right where the model needs it. A rule of thumb that works: if you find yourself writing a sentence of prose explaining what a field means, the field has the wrong name. Rename it and delete the sentence.

Fix the order, and put evidence first. Two payoffs. The dull one is that a stable key order makes diffing two runs trivial. The useful one is that the model generates left to right, so a field produced early is in the context when the later fields are produced. Put evidence_quote before classification and the label is generated with the quote already on the page — the same mechanism as lesson 5, wearing a different hat. Be honest about the size of the effect: the direction is a straightforward consequence of how generation works, but how much it helps is task-dependent. It costs you nothing to order the fields well, so do it, then measure whether it mattered.

Define what unknown looks like. This is the field that stops invention. A model asked for a notice period that is not in the contract will produce a plausible one, because plausible text is the only thing it makes. Give it a legitimate way to say nothing, and say when to use it:

• Pick one representation and enforce it. null for absent — not an empty string, not "N/A" alongside "n/a" alongside "unknown".
• Write the rule explicitly: "Use null when the value is not stated in the document. Never infer it from context."
• If unknown is a category, put it in the enum: [approve, reject, unknown]. A four-value decision with no escape hatch is a three-value decision plus a guess.

A schema without a defined unknown is a schema that asks the model to make something up.

from anthropic import Anthropic

client = Anthropic()

RECORD_INVOICE = {
    "name": "record_invoice",
    "description": "Record the fields of exactly one invoice.",
    "input_schema": {
        "type": "object",
        "properties": {
            # Evidence first: it is generated before the values it supports.
            "source_quote": {
                "type": ["string", "null"],
                "description": "The line the total was read from, verbatim. Null if not found.",
            },
            "invoice_number": {"type": ["string", "null"]},
            "issue_date": {
                "type": ["string", "null"],
                "description": "YYYY-MM-DD. Null if the invoice does not state it.",
            },
            "total_minor_units": {
                "type": ["integer", "null"],
                "description": "Cents. 1204.00 USD is 120400. Never a float.",
            },
            "currency": {"type": ["string", "null"], "enum": ["USD", "EUR", "GBP", None]},
        },
        "required": [
            "source_quote", "invoice_number", "issue_date",
            "total_minor_units", "currency",
        ],
    },
}

resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=500,
    temperature=0,
    system="Read the invoice. Use null for anything not stated. Never infer a value.",
    tools=[RECORD_INVOICE],
    tool_choice={"type": "tool", "name": "record_invoice"},   # not optional
    messages=[{"role": "user", "content": "<invoice>...</invoice>"}],
)

block = next(b for b in resp.content if b.type == "tool_use")
print(block.input)      # already a dict - no fence to strip, no regex

Nothing in that prompt asks nicely for anything. Every rule that could have been a sentence is a field name, a type, an enum or a null. The system prompt is down to one line, and it only exists to say what to do about missing values — which is the one rule a JSON Schema cannot express on its own.

The shape is now guaranteed. Whether the values are right is a different question, and it is what Part 3 is about.