Citations and grounding

1.An id is a handle

Put an id next to each passage and ask the model to cite it. Start with the ids, because a bad id makes everything downstream useless.

The test: six months from now, someone pastes a cited id into a search box. Does it find the exact passage? If not, the citation is decoration.

Store the id with the chunk at index time, beside the metadata from Metadata and filtering, and carry it through retrieval and reranking untouched. It is the same string in your index, prompt, logs and UI.

The prompt format only needs to be easy to parse, because your verifier will parse it:

[refund-policy-digital] Refunds on digital goods are available within 14 days of purchase.

[refund-policy-physical] Physical goods may be returned within 30 days if unopened.

A citation is not a promise from the model. It is a handle you can pull on in code.

2.The prompt, including permission to fail

Three instructions do most of the work, and the third is the one people leave out.

Answer only from the passages below.

After each claim, cite the id it came from, and quote the words you used. The quote is what makes checking possible. Without it you can confirm the id exists; with it you can confirm the id supports the words.

If the passages do not answer the question, say exactly that and stop. A model with no permitted way to fail will not fail - it will produce something. Naming the failure output, with exact words, turns confident invention into a signal you can count and alert on.

The last row matters most: a model that takes half a fact from each of two chunks and cites both has two valid citations and a claim that is in neither.

The failure branch has two cases beyond a plain miss. First, the set can be empty. A live permission check, as in Permissions that travel with the data, or a metadata filter can remove every passage. The only correct output is then the failure line; code that quietly retries without the filter turns a refusal into a leak. Second, near-miss passages make abstaining harder. Near-miss passages hurt models unevenly, as Chroma's vendor study found, so test yours and count the refusals; Measuring the whole answer shows how.

Let the API do the pointing

As of October 2026, Anthropic's Messages API can return citations itself. You mark a document block, or a search_result block whose source is your chunk id, with "citations": {"enabled": true}, and the text blocks of the reply carry a list of citations.

The docs say the API parses the citations, so each pointer is guaranteed valid, and cited_text does not count toward output tokens. That removes both the regex and the output cost of typed quotes. Two limits from the same docs: citations with structured outputs return a 400 error, and a scanned PDF with no extractable text cannot be cited. A valid pointer is still only a pointer, so the rest of this lesson applies unchanged.

3.Verifying, and what it cannot tell you

Here is the check for the prompt-based route. It splits the answer into sentences, pulls each quoted span and the id after it, and confirms the words are in that chunk. The answers are hand-written, so no model produced this output.

import re

chunks = {
    "policy-14": "Refunds on digital goods are available within 14 days of purchase.",
    "policy-15": "Physical goods may be returned within 30 days if unopened.",
}

def normalise(text):
    return re.sub(r"\s+", " ", text).strip().lower()

def verify(answer):
    results = []
    for sentence in re.split(r"(?<=[.!?\]])\s+(?=[A-Z])", answer):
        cites = re.findall(r'"([^"]+)"\s*\[([^\]]+)\]', sentence)
        if not cites:
            results.append(("UNCITED", sentence))
        for quote, chunk_id in cites:
            chunk = chunks.get(chunk_id)
            if chunk is None:
                results.append(("UNKNOWN_ID", chunk_id))
            elif normalise(quote) not in normalise(chunk):
                results.append(("BAD_QUOTE", f"{chunk_id}: {quote}"))
            else:
                results.append(("OK", chunk_id))
    return results

print(verify('Digital goods: "available within 14 days of purchase" [policy-14].'))
print(verify('Digital goods: "refundable for 14 days" [policy-14].'))
print(verify('Opened items: "returned within 60 days" [policy-99].'))
print(verify('Digital goods: "available within 14 days of purchase" [policy-14]. Physical returns are free.'))
# -> [('OK', 'policy-14')]
# -> [('BAD_QUOTE', 'policy-14: refundable for 14 days')]
# -> [('UNKNOWN_ID', 'policy-99')]
# -> [('OK', 'policy-14'), ('UNCITED', 'Physical returns are free.')]

Normalising whitespace and case matters, because models reflow text. For heavily formatted corpora, strip punctuation too, or use a fuzzy ratio with a high threshold.

What to do on a failure is a product decision, and the rate is worth counting. Here is one invented day of logs, 400 answers.

Retrying the 33 blocked answers once adds 33 calls to 400, about 8 percent more. The check is a string comparison, so it adds no model call. The quotes cost output: five claims with 20-word quotes add about 100 words to every answer, which is the argument for the built-in cited_text, as it costs no output tokens. Track the 88 percent: it moves when you change the prompt, the model or the chunk size, and a drop is worth an alert.

Citations are also a feature. Show the cited passage beside the answer, as a link or a side panel, and the reader can check you instead of trusting you. Bug reports then arrive with the passage attached.

The failure to name: the true quote, the wrong claim

Verification proves one narrow thing: the quoted words exist in the cited chunk. Suppose policy-14 really says refunds on digital goods are available within 14 days of purchase, for members only. The model answers "Digital goods are refundable within 14 days" and quotes "within 14 days of purchase" [policy-14]. Every quoted word is present, the check passes, and a non-member is told something false.

Three habits narrow the gap. Ask for the whole sentence in the quote, so the qualifier sits beside the claim. Treat cross-passage claims, heading-only passages and superseded documents as known holes: all pass the check, and the last is a metadata problem. For high-stakes answers add a second check: a model given one claim and its passage, asked whether the passage supports it. That is one call per claim, so 5 claims on 400 answers is 2,000 calls, and the judge is noisy; Measuring the whole answer shows how to measure it.

How big is the gap? An April 2023 audit of four commercial generative search engines found that, on average, 51.5 percent of sentences were fully supported by their citations and 74.5 percent of citations supported their sentence. Those engines are old, so read the numbers as the shape of the problem. Retrieval matters too: in one August 2025 agent study (BrowseComp-Plus), swapping BM25 for a stronger embedder raised every tested model's citation precision.

Page images break the quote check, because there is no text to match. Cite the page number and show the page. Do not promise region-level pointers: models boxing the supporting region did very poorly in ViDoRe V3. Retrieving pages as images covers the rest.

When not to use it. Skip the quote check when nobody will look: a low-stakes brainstorm gains nothing from the extra output and the blocked answers. It fits badly when an answer is a computation or a summary of many passages, because no single span supports it; cite the source ids and let a person or a judge read it. If you need strict JSON with Anthropic's built-in citations, you cannot have both as of October 2026, so put the ids in a field and verify them yourself.

What you have bought is the removal of the crudest failure, text with no source, and every remaining answer is traceable to a passage a human can read in seconds. That is not correctness. Correctness needs the evaluation in Measuring the whole answer, and for anything consequential it still needs a person.