Junior interview questionAsked by candidates · YouTube
Why is prompt injection possible at all, and why can't it be fixed the way special-token injection is?
Model answer
Instructions and data share one token stream; role markers are just more tokens.
Scroll sideways to see the whole diagram
An application's context comes from several places, and different people wrote each part.
Text version
- An application's context comes from several places, and different people wrote each part.
- Before the call, all of it is concatenated into one prompt and turned into tokens. Role markers are tokens too.
- The model reads one sequence. A sentence in a web page that reads like an instruction looks the same as one in your system prompt.
After AI Engineering, Ch. 5, "System Prompt and User Prompt" and "Model-level defense". Illustrative parts.
Special-token injection has a clean fix: control tokens are a fixed set you refuse to encode from untrusted text. An instruction is ordinary words, so nothing at the token level marks it. Models are trained to follow instructions, an attacker's too.
Practise the rest
12 Prompt injection questions, 492 in all, each with a model answer. Free.
More Prompt injection interview questions
- What is the difference between direct and indirect prompt injection? Give an example of each.
- Your assistant has no tool that sends anything. Can an injected web page still leak the user's data?
- You're designing an email assistant that reads, summarises and forwards mail. Which defences do you put in, and where?
- A reviewer says: “The system prompt forbids following instructions in emails, so the forward tool is safe, and our red-team run had zero violations.” Do you accept that?
- What's the difference between a jailbreak and a prompt injection?
- Do XML tags or delimiters around untrusted text stop prompt injection?