Retrieval as a tool the agent calls

1.Two ways to get context to the model

Every lesson so far retrieved up front. Your code rewrites the query, searches, reranks and hands the model a fixed set of chunks, and the model never chooses. The other style is just in time. The model starts with the question and a few light handles - file paths, document ids, saved queries - and calls tools to load what it needs as it goes. Anthropic's engineering essay on context (September 2025) describes it this way and says its coding tool, Claude Code (as of September 2025), works like that, with commands such as grep and file reads.

Iterative retrieval covered a loop you write, with a decide step and a cap. Here the loop belongs to the model, and your code supplies the tools and the fence. How to design a tool and how the loop itself runs are the Agents course: Designing a tool a model can actually use and A loop, not a brain.

Most real systems mix the two: a little context up front, such as a project summary, and the rest on demand. The essay notes that cues such as file names and dates help the agent judge how and when to use a file. A wiki with titled, dated pages gives it names to search for; a flat pile of untitled chunks gives it nothing. The essay also admits the cost: "runtime exploration is slower than retrieving pre-computed data."

Up front, your code decides what the model sees. Just in time, the model decides, and your code decides what it is allowed to reach.

2.The model sets the pace, and the retriever still decides

How many searches will the agent run? Not a number you set. A study from August 2025, BrowseComp-Plus, fixed a corpus of about 100,000 web pages and 830 very hard multi-clue questions, so agent and retriever could be varied one at a time. GPT-5 (as of August 2025) averaged over 20 searches per question. Two small open models averaged under one and under two, although they were told to search. Raising the reasoning effort of an open-weight model, gpt-oss-120B (as of August 2025), moved it from about 2 searches to about 18. Rounds are a property of the model and its setting. These are puzzle-like questions, so read the direction, not the counts.

Retrieval quality still decides the outcome. Swapping BM25 for a dense retriever, Qwen3-Embedding-8B (as of August 2025), raised GPT-5's accuracy from 55.9% to 70.1% and trimmed its searches from 23.2 to 21.7. A smaller model, GPT-4.1 (as of August 2025), went from 14.6% to 35.4%. Given only the labelled evidence documents, it reached 93.5%, so finding documents, not reading them, was the bottleneck. The study tested neither hybrid fusion nor a reranker, so it says a better index helps, not by how much. Keep Hybrid search and Reranking behind the search tool.

3.Tools, a trace, and the fence

The evidence for which tools to offer is mixed, and mostly from preprints and vendors:

Read together, the evidence says keyword tools alone may be enough on some corpora. The AWS paper states no corpus sizes, so take it as a reason to try the cheap version first. Add semantic search when the corpus is large or the wording varies. In Cursor's online test the code-retention gain was 0.3% overall and 2.6% on repositories of 1,000 or more files, which is why you should segment your results.

Offer lexical and semantic search as two tools, not one. Exact names, error codes and identifiers suit lexical search, plain-words questions suit semantic search, and the agent picks per query. One fused hybrid search is a fine first tool. Add a read tool. In BrowseComp-Plus results were cut to 512 tokens, and a tool that fetched whole documents lifted GPT-4.1 from 35.4% to 43.6%, while a small open model barely used it. Return lean results, and check permission inside the tool, never in the prompt.

# The tools only. The loop that calls them belongs to the Agents course.
# `index` stands for your existing hybrid + rerank pipeline.
def search_tool(mode, use):
    return {"name": f"search_{mode}",
            "description": use + " Snippets are cut off: call read for the full text.",
            "input_schema": {"type": "object", "required": ["query"], "properties": {
                "query": {"type": "string"},
                "k": {"type": "integer", "minimum": 1, "maximum": 10}}}}

SEARCH_LEXICAL = search_tool("lexical", "Exact names, error codes and identifiers.")
SEARCH_SEMANTIC = search_tool("semantic", "Questions in plain words.")
READ_TOOL = {
    "name": "read",
    "description": "Read lines start to end of one document by id, at most 120 lines.",
    "input_schema": {"type": "object", "required": ["doc_id", "start", "end"], "properties": {
        "doc_id": {"type": "string"}, "start": {"type": "integer"}, "end": {"type": "integer"}}},
}

# `user` and `seen` come from your code, never from the model's arguments,
# so the model cannot widen its own access.
def search(mode, query, k=5, *, user, seen):
    hits = index.search(query, mode=mode, k=min(k, 10), groups=user.groups)
    fresh = [h for h in hits if h.id not in seen]
    seen.update(h.id for h in fresh)
    if not fresh:
        return "No new results. Change the query or answer from what you have."
    return "\n".join(f"{h.id} | {h.title} | {h.updated:%Y-%m-%d}\n  {h.text[:240]}..." for h in fresh)

def read(doc_id, start, end, *, user):
    doc = index.get(doc_id, groups=user.groups)
    if doc is None:
        return "Not found."
    start = max(start, 1)
    return "\n".join(doc.text.splitlines()[start - 1 : min(end, start + 119)])

Here is an invented trace for "why does checkout fail for gift cards?" over a wiki and a code repository.

What it costs. Five calls look cheap, but every turn re-reads everything gathered so far. Assume 500 tokens for the question and instructions and 800 per tool result (both invented). Turn n reads 500 + 800 x (n - 1) tokens, so six turns read 15,000 in all, against about 1,500 for one search that sends five 200-token chunks. A run of 25 near-duplicate calls takes 26 turns and reads about 273,000 tokens, 18 times the trace above. Prompt caching can discount the repeated prefix; measure your own bill. Why the bill exploded covers the pattern.

Two named failures. The agent that never reads: five searches, no read call, and a confident answer built on a snippet cut mid-sentence. Say in the tool description that snippets are cut off, and count read calls per run. The agent that never stops: 25 near-duplicate searches. The seen set above turns repeats into "No new results", the thrash check from Iterative retrieval moved into the tool. Add a hard cap, say 12 rounds or 80,000 tokens read in total (invented; 12 rounds make 13 turns and read about 69,000 here, so the token cap guards bigger results). On a question nobody wrote down, such as "which customers will the next pricing change affect", the run ends at round 12 with "not found" and the queries it tried. Clear old results as the run grows: What the loop remembers within one run covers it, and Planning a run, and knowing when to stop covers the budget.

Measuring it. Evaluate offline on a frozen corpus, as BrowseComp-Plus does, so a score change is the agent's and not the data's. Online, watch a downstream signal and split by corpus size or question type, since the average hides the lift. Measuring retrieval and Measuring the whole answer cover the method.

When not to use it. Skip it when one search answers the question, as in an FAQ lookup: the pipeline is cheaper and faster. Skip it on latency-bound paths, where every round is a model call plus a search. Skip it when your corpus has no names or structure to steer by, because exploration then spends context on dead ends. Skip it when you need predictable cost. When RAG is the wrong tool places this among the other options.