How a model picks a tool

Function calling looks like a plugin system from the outside: you register functions, the model uses them. It is not one. Once you see what actually happens the failure modes stop being mysterious.

1Your tools become text

Send a request with a tools array and the API does something plain: it builds a system prompt out of your tool definitions and prepends it to the conversation. Anthropic publishes the scaffold. It opens by telling the model it has access to a set of tools, adds formatting instructions and a note that string and scalar parameters go in as-is while lists and objects use JSON, then says here are the functions available in JSONSchema format and pastes your definitions in. Your own system prompt goes after that, and the tool configuration after that.

Three consequences fall straight out of this.

Tool definitions cost input tokens, on every request. Names, descriptions and schemas are all prompt text, plus a few hundred tokens of scaffold. The exact scaffold cost varies by model and by tool_choice; there is a table on the tool use overview page — look it up rather than trusting a number in a tutorial, this one included. The practical point is that a forty-tool array is a fixed tax paid on step one and again on step thirty of the same run.

Prompt caching applies to it. The tools array sits at the front and rarely changes, which makes it the ideal thing to cache across the steps of a long run. Changing tool_choice mid-run invalidates cached message content, so pick it and leave it alone.

Descriptions are read the way instructions are read. They are not documentation sitting next to the model. They are in the prompt, competing for attention with everything else in the prompt.

Here is what the model does and does not have:

Read the right-hand column when a tool call surprises you. Almost every "why did it do that" is something in that column that you assumed was in the left.

2Tool, or no tool

With the default tool_choice of auto, the model makes two decisions in order, and they fail for different reasons. First: does this turn need a tool at all? Second: which one?

The first one is a judgement about whether the answer is already available. Stable knowledge, creative work and ordinary conversation get answered directly. A request that maps onto a described capability, and whose answer is not already in context, gets a tool call. That boundary is soft, which is exactly why your get_order_status tool sat there while the model wrote a plausible paragraph — the model treated the question as answerable from what it knew.

You have two ways to move that boundary, and they are not interchangeable.

Two things to know before you reach for the forced options. Forcing a tool prefills the assistant turn, so you lose the natural-language sentence the model would have written first — if you want both the explanation and the call, stay on auto and ask for the tool in the user message instead. And forced tool use is not universally supported: some models and some thinking settings reject any and tool with a 400. Check the current support table before you build on it.

3Which tool, and why the description wins

Now the second decision. The model is matching the request against the text of each definition and picking the best fit. It has your description, your parameter names and your types. It has nothing else.

This is why the description does more work than the schema. Types tell the model what a legal argument looks like. Only the description tells it what the tool is for, and selection is a question about purpose.

Anthropic's own guidance is blunt about it: extremely detailed descriptions are by far the most important factor in tool performance, and they suggest three or four sentences minimum. A description that says "Gets the stock price for a ticker" leaves the model guessing about the exchange, the currency, whether it is live or delayed, and whether it also returns volume. A description that says all of that, plus "use this when the user asks about the current or most recent price of a specific stock" and "it will not provide any other information about the company", removes every one of those guesses.

The second failure is overlap. Nothing checks that your descriptions carve the space up cleanly. Ship search_docs and search_knowledge_base with descriptions that could each be either, and the model resolves the tie on wording — consistently, and consistently wrong for half your traffic. The fix is not a better model, it is one description that says what the other one does not cover.

The model never sees your function. It sees your sentence about your function. Treat every selection bug as a description bug until you have ruled that out.

The way to settle an argument about this is to run it. Two definitions, same task, one call:

import anthropic

client = anthropic.Anthropic()

vague = {
    "name": "get_order_status",
    "description": "Gets order status.",
    "input_schema": {
        "type": "object",
        "properties": {"order_id": {"type": "string"}},
        "required": ["order_id"],
    },
}

specific = {
    **vague,
    "description": (
        "Look up the live delivery status of a single customer order by its order ID. "
        "Use this whenever a customer asks where their order is, whether it has shipped, "
        "or when it will arrive - never answer those from general knowledge. "
        "Returns the current status, the carrier and the estimated delivery date. "
        "It does not return line items, prices or refund history."
    ),
}

for label, tool in [("vague", vague), ("specific", specific)]:
    resp = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=512,
        tools=[tool],
        messages=[{"role": "user", "content": "Where is my order? It's A-91422."}],
    )
    called = [b.name for b in resp.content if b.type == "tool_use"]
    print(f"{label:9} stop_reason={resp.stop_reason:10} called={called or 'nothing'}")

Run it against your own tools with your own real user phrasings. When the two rows differ, you have found the cheapest fix in this course. Next: how to write the schema underneath that description so the arguments are right the first time.