Thinking and effort, model by model

Base, instruct, and reasoning models said that thinking is a mode on an ordinary model, chosen per request. This lesson covers the settings that choose it, which are not the same from one model to the next. Everything about a named model below is as of Oct 2026 and comes from the provider's own documentation, so read the tables as a guide to what to look up, not as a fixed list.

1.The default is not shared

A request with no thinking field means different things on different models. This is what Anthropic's thinking page says, as of Oct 2026, for five of its models:

Adaptive thinking means the model reads each request and decides for itself whether to think and for how long. You can read the table as a trap more than a reference. The quiet failure is the migration where you change the model id, send the same request, and thinking now runs.

Same field, same name, different default. Read the row for the model you are calling, not the one you used last year.

The other two big providers expose the same idea under other names. OpenAI's reasoning guide, as of Oct 2026, uses reasoning.effort with values from none to max, and says the values a model accepts depend on the model. It lists at least one model where none returns a 400, and a default of medium for the models it names. Google's thinking page, updated 14 Oct 2026, uses thinking_level with minimal, low, medium or high depending on the model. It marks thinking as on by default for nearly every model, and documents no way to switch it off on those. I read each page once and did not test either, and the Google page I read mainly covers its Interactions API.

2.Two dials, and a bill you cannot see

On Anthropic, thinking decides whether the model may think up front, and effort decides how hard it works on the whole reply. You set effort at output_config.effort, with five levels as of Oct 2026: low, medium, high, xhigh and max. As of Oct 2026, the default is high on Sonnet 5.5 and medium on Opus 5.5 and Haiku 5.5, so a request that sets nothing runs at a different level on each.

Effort is soft guidance, not a token count. Anthropic says that at low effort the model still thinks on hard problems, only less. It also reaches past thinking: text and tool calls get terser too.

The old dial was a fixed number, thinking: {"type": "enabled", "budget_tokens": N}. As of Oct 2026, on the newest models it returns a 400 that tells you to use adaptive thinking and effort. Anthropic says there is no fixed mapping from a budget to an effort level, so you cannot convert; run your tests at two or three levels instead.

Turn thinking down with effort before you try anything else. As of Oct 2026, Anthropic reports that asking Sonnet 5.5 in the system prompt to think less does not reliably work. Also as of Oct 2026, changing top-level effort between requests breaks prompt caching, so pick one level per conversation. As of Oct 2026, per-message effort (beta) on Sonnet 5.5, Opus 5.5 and Haiku 5.5 keeps the cache. Caching and batch: the two big discounts explains why.

Now the bill. As of Oct 2026, on Sonnet 5.5 the thinking text comes back empty by default, but you are billed for the full thinking, as output tokens, and it counts toward max_tokens. The response reports how many in usage.output_tokens_details.thinking_tokens. As of Oct 2026, OpenAI does the same under the name reasoning_tokens, and Google bills its thought tokens too.

That sharing is also how the cut-off answer happens. You set max_tokens to 500, which was plenty for the old answers. On a hard request the model can spend all 500 on thinking and stop with stop_reason: "max_tokens" and little or no text, and you still pay. Anthropic gives two remedies: raise the cap, or lower the effort if the request did not need the thinking. As of Oct 2026, OpenAI describes the same failure as a response with status incomplete and the reason max_output_tokens, sometimes before any visible output. A second break comes with the first block: code that reads content[0].text fails when the first block is a thinking block. Read blocks by their type.

Last, comparisons. If one model defaults to high and the other to medium, running both on defaults compares two amounts of thinking as well as two models. Fix effort on both. One August 2026 preprint on three OpenAI model upgrades did this, holding effort at medium, and also found that answer length changed between versions (Xu and Wu; not peer reviewed). The Evals course lesson Regression testing a prompt or model change covers how to compare properly.

3.Finding the lowest setting, and when to skip thinking

The rule from Base, instruct, and reasoning models still holds, now with real settings. Begin at the lowest setting your task tolerates, find the requests that fail, and raise it one step at a time. Keep a step only if a test set says it helped. The Evals course lesson Building an eval set from your own traffic shows how to build that set.

Some work should never leave the bottom of that loop. A classifier, an extractor and a rewrite have no intermediate steps to get wrong. As of Oct 2026, OpenAI's guide names classification and latency-critical work as the cases for none, and Anthropic says low skips thinking for simple tasks. Both are vendor advice, so test it. As of Oct 2026, Anthropic also says no effort level guarantees a thinking block on every request, in either direction, and that in a tool loop the follow-up requests can skip thinking even at xhigh and max. Research on inference-time compute (Snell and colleagues, 2024) found that sharing extra compute out by prompt difficulty beat a best-of-N baseline, by more than 4x in efficiency. The lesson carries over: spend more only where it helps.

This advice does not apply everywhere. As of Oct 2026, on older models such as Sonnet 4.5, a fixed budget_tokens is still the only control. On models you host yourself, how thinking is exposed depends on the model and the server, and those were not checked here. And more thinking is not a cure for a missing fact: Why models hallucinate explains why a model that does not know something will still produce an answer.

The script below puts three tasks through the lowest setting and through high, and prints where the tokens went. It asks the Models API (as of Oct 2026, through capabilities.thinking.types) which off switch the model accepts: disabled where that is allowed, between_tools where it is not. If you swap in a model that is always on, neither works, so use adaptive thinking at low there.

from anthropic import Anthropic

# Check the models page for current ids; as of Oct 2026 this one is valid.
MODEL = "claude-sonnet-5-5"

client = Anthropic()

TASKS = {
    "label": "Answer with one word, billing, bug or other: 'I was charged twice this month.'",
    "puzzle": (
        "Alice, Bram and Chen each need one of the three one-hour slots at 9, 10 and 11, "
        "in one room. Alice cannot take 9. Chen cannot take 11. Bram must come before Chen. "
        "Give one valid schedule."
    ),
    "extract": "Reply with only the invoice number: 'Invoice INV-20418, due 3 Nov, total 412.50 GBP.'",
}


# Ask the Models API how this model switches thinking down, instead of guessing.
types = client.models.retrieve(MODEL).capabilities.thinking.types
OFF = {"type": "disabled"} if types.disabled.supported else {"type": "between_tools"}
SETTINGS = {
    "lowest": (OFF, "low"),
    "high": ({"type": "adaptive"}, "high"),
}


def run(prompt, thinking, effort):
    response = client.messages.create(
        model=MODEL,
        max_tokens=16000,  # thinking and the answer share this one cap
        thinking=thinking,
        output_config={"effort": effort},
        messages=[{"role": "user", "content": prompt}],
    )
    usage = response.usage.model_dump()
    used = (usage.get("output_tokens_details") or {}).get("thinking_tokens")
    text = next((b.text for b in response.content if b.type == "text"), "")
    return usage["output_tokens"], used, response.stop_reason, text.strip()


for name, prompt in TASKS.items():
    for label, (thinking, effort) in SETTINGS.items():
        total, used, stop, text = run(prompt, thinking, effort)
        print(f"{name:8}{label:7} output={total:5} thinking={used} stop={stop} {text[:50]!r}")

You will see six lines, one per task and setting, each with the billed output tokens, the thinking share of them, and the stop reason. Expect the label and the extraction to cost about the same on both lines, and the puzzle to be where high spends more. No numbers are printed here because they depend on your prompts and the model, and effort is soft guidance. The request was built with the current SDK and checked against the shape Anthropic documents, but no live response was seen, so your first run is the first measurement. If thinking prints None, your SDK or model does not report the breakdown. If stop reads max_tokens, raise the cap. Swap in your own three tasks before you decide anything.