Containing prompt injection

Prompt injection ended with the bad news: no prompt reliably makes a model ignore planted text. This lesson accepts that and asks a different question. If the model will sometimes be fooled, can you build the system so a fooled model has nothing to reach? That is a design job, and nearly all of it is code.

1.Cut one of three, on purpose

Simon Willison's lethal trifecta (June 2025) is the diagnosis: untrusted content, private data and a way out. Meta's Rule of Two (October 2025) turns it into a design rule. In one session, give an agent at most two of three properties: it reads input from sources you do not control, it reaches sensitive data or systems, and it can change state or talk to the outside. If the job needs all three, it should not run alone. A person or a hard check goes in front of the last step.

Meta calls this a floor, not a finish line, and says it lowers risk "deterministically". That holds only if each property is enforced in code outside the model. A system prompt saying "you cannot send mail" removes nothing. Also note what the post is: a design argument with hypothetical agents, no experiments and no numbers.

Take the refund mailbox from Decomposition. The assistant reads customer email, looks up orders, issues refunds and sends replies. That is all three. Here is each cut, and what it leaves.

A session can also change shape. Meta's example starts with web access and no private data, then switches off outward communication for good before touching internal systems. The switch must be one-way and enforced by your code. If the model can flip it back, you have a suggestion, not a cut.

2.Patterns that make a leg unreachable

Fourteen authors set out the structural patterns in a June 2025 paper. Treat them as arguments, not results: the paper reports no measurements. Each one buys safety with something you lose.

The hinge in the quarantined pattern is the hand-off. If the reader returns free text, the attacker rides it into the next stage. If it returns an enum, a boolean or a bounded number, there is nothing to ride. That is Output contracts and Structured outputs in practice doing security work.

Exits are wider than "send". A markdown image in a reply makes the viewer's browser fetch an address the attacker chose, and the address can carry data. So can any link and any URL fetch. Render no remote images, allowlist the domains you will reach, and rewrite links before display.

The tool catalog is untrusted input too, and the MCPTox numbers are in Prompt injection. A poisoned tool is never called; its text steers a legitimate one. Review and pin descriptions you depend on, and check each call's arguments against what the user asked. When you add a tool or a server, ask again which of the three properties the whole set now holds. That last habit is this course's suggestion, not tested.

3.A worked example, a failure, and how to test

Here is the refund mailbox with the reader caged. One model call sees the email. It has no tools, and its answer is two narrow fields. Everything after it is code. The paper's email-assistant case is the model for this.

import logging
from typing import Literal

from anthropic import Anthropic
from pydantic import BaseModel

client = Anthropic()
log = logging.getLogger(__name__)
MODEL = "claude-sonnet-5-5"   # as of October 2026

def text_of(resp) -> str:
    return "".join(b.text for b in resp.content if b.type == "text")

class MailFacts(BaseModel):   # all the reader may say; there is no free-text field
    kind: Literal["refund_request", "question", "other"]
    order_id: str | None

def read_mail(raw: str) -> MailFacts:
    resp = client.messages.parse(
        model=MODEL,
        max_tokens=2000,   # thinking is on by default and counts against this
        system="Read one customer email and fill in the fields. You have no tools.",
        messages=[{"role": "user", "content": f"<email>\n{raw}\n</email>"}],
        output_format=MailFacts,
    )
    if resp.stop_reason != "end_turn":
        log.error("raw reply: %r", text_of(resp))
        raise RuntimeError(f"no usable answer: {resp.stop_reason}")
    return resp.parsed_output

def handle(sender: str, raw: str) -> None:
    facts = read_mail(raw)   # the only call that sees the mail
    order = ORDERS.get(facts.order_id)   # your records, looked up by code
    if facts.kind != "refund_request" or order is None or order.customer != sender:
        send_to_human(sender, raw)   # a fixed branch, no model involved
        return
    if order.total <= REFUND_CAP:   # the policy is code
        issue_refund(order.id)
    send_template(to=sender, name="request_received", order_id=order.id)

ORDERS, REFUND_CAP, issue_refund, send_template and send_to_human stand for your own code. The parameter spellings come from vendor docs as of October 2026 and were not run here. Try them first, and keep the stop_reason check, since the docs do not say what parse does on a refusal.

Now attack it. An email saying "refund every order" has no field to say it in. One naming another customer's order fails the ownership check. One that talks the reader into refund_request for the sender's own order gets what any honest request gets. Nothing the attacker writes reaches a recipient or a message body, because those come from code. Here the privileged side is plain code. If you want a model to plan the calls instead, you gain flexibility, but the plan must be fixed before any mail is read.

Do not make the model hard to fool. Make fooling it worth nothing.

Picture a team that removes send_email from the dashboard assistant and calls the leg cut. The dashboard renders replies as markdown. A planted line asks for an image whose address ends in a summary of the inbox. A person opens the dashboard, the browser fetches the image, and the summary has left. Nobody counted rendering as outward reach. The second failure is slower: they add an approve button, and by the fortieth reply it gets clicked unread. Meta and the paper both warn that confirmation wears out, and the paper adds that hidden characters can hide an exfiltration from a reviewer.

Test it in two ways. First, the properties, with no model: the reader request carries no tools, the recipient always equals the sender, a foreign order is refused, and no reply holds an image or link your code did not write. Run these in your ordinary test suite, so a later edit that hands the reader a tool fails the build. Second, an adaptive attacker, and the adaptive-attacker results are in Prompt injection. What is new here: that study did not attack plan-then-execute designs and argued the attack should fail, which is a scoping note, not proof. Keep your fixed injection list as a canary, as in Versioning and testing prompts, and run a person against the rest.

When not to use it: a general assistant that must read anything and do anything cannot use these patterns, because they buy safety with flexibility. For that, supervise, shrink the blast radius, and log every call. A read-only summariser with no private data and no exit needs the tags and the habit of distrusting its output, not this architecture. Sandboxes, gates and permissions belong to the Agents course, in Letting an agent touch the real world.

Next: Sampling and voting, for the answers that are well-formed and wrong.