No defence is complete, and wrap-up

Predict first

You enable the instruction hierarchy, delimit all untrusted text and scan it for attack phrases. How many steps of the two planted attacks are now blocked?

All of themHalf of themOnly the last stepNone: they make some steps less likely, and nothing blocks the tool call or the link

Which defence breaks which step?

Lesson-design mapping · 6 defences · 2 attacks · not a measurement

Model trained on an instruction hierarchy model levelTool outputs rank below the system prompt and the user, so conflicting instructions in them should lose.Delimit and label untrusted text prompt levelWrap every tool result in marked tags and tell the model that text inside them is data, never instructions.Scan untrusted text for attack phrases system levelFlag known patterns such as "ignore previous instructions". A reworded payload passes.Least privilege for tools system levelThis task needs reading and search only: no forward_email, and send_email limited to Dana's contacts.Human confirms irreversible actions system levelEvery send or forward is shown to Dana and runs only if she approves it.Check the output before showing it system levelRemove links to domains not on an allowlist, and don't auto-load images or previews.

Email asks to forward the inbox

1. The email's text is read into the contextless likely by scan untrusted text for attack phrases
2. The model treats the planted line as an instructionless likely by model trained on an instruction hierarchy, delimit and label untrusted text
3. The model calls forward_email to an outside address
4. Ten emails reach records@attacker.example

Harder, not stopped: every step can still happen

Web page asks for a link carrying the email

1. The page's hidden text is read into the contextless likely by scan untrusted text for attack phrases
2. The model treats the hidden comment as an instructionless likely by model trained on an instruction hierarchy, delimit and label untrusted text
3. The reply contains a link with the email in its address
4. Opening or previewing the link sends the email to the page's owner

Harder, not stopped: every step can still happen

The mapping is fixed by this lesson, following AI Engineering Ch. 5 and OWASP LLM01:2025. “Less likely” means the defence depends on the model complying, which can fail; “blocked” means code outside the model stops the step. Illustrative context written for this lesson, not a real product and not a model output. Addresses use the reserved .example domain.

Quick check

A reviewer says: 'Our system prompt tells the model never to follow instructions in emails, so the forward tool is safe.' What's the best reply?

Agreed: the instruction is explicitAdd a second copy of the instruction at the endRemove the system promptThe model can still fail to comply; gate forwarding with least privilege or user confirmation

Predict first

You enable the instruction hierarchy, delimit all untrusted text and scan it for attack phrases. How many steps of the two planted attacks are now blocked?

All of themHalf of themOnly the last stepNone: they make some steps less likely, and nothing blocks the tool call or the link

Which defence breaks which step?

Lesson-design mapping · 6 defences · 2 attacks · not a measurement

Email asks to forward the inbox

1. The email's text is read into the contextless likely by scan untrusted text for attack phrases
2. The model treats the planted line as an instructionless likely by model trained on an instruction hierarchy, delimit and label untrusted text
3. The model calls forward_email to an outside address
4. Ten emails reach records@attacker.example

Harder, not stopped: every step can still happen

Web page asks for a link carrying the email

1. The page's hidden text is read into the contextless likely by scan untrusted text for attack phrases
2. The model treats the hidden comment as an instructionless likely by model trained on an instruction hierarchy, delimit and label untrusted text
3. The reply contains a link with the email in its address
4. Opening or previewing the link sends the email to the page's owner

Harder, not stopped: every step can still happen

Quick check

A reviewer says: 'Our system prompt tells the model never to follow instructions in emails, so the forward tool is safe.' What's the best reply?

Agreed: the instruction is explicitAdd a second copy of the instruction at the endRemove the system promptThe model can still fail to comply; gate forwarding with least privilege or user confirmation

What to keep

No defence is complete, and wrap-up

What to keep