Prompt anatomy and chat templates interview questions
5 questions with model answers
- junior 1
- mid 2
- senior 2
5 prompt anatomy and chat templates questions asked in AI engineer, ML engineer and applied AI engineer interviews, each with a model answer you can say out loud in a minute or two: 1 junior, 2 mid-level and 2 senior. The lesson Anatomy of a Prompt: Roles, Templates and Instruction Hierarchy teaches the ideas behind them.Updated
Model answer
One token sequence. The chat template writes each role as marker tokens around its content; AI Engineering: system and user prompts are "processed the same way".
Scroll sideways to see the whole diagram
An API takes a list of messages, each with a role. This is the book's system and user prompt.
Text version
- An API takes a list of messages, each with a role. This is the book's system and user prompt.
- Before the model sees anything, a chat template turns the list into one sequence.
- Llama 3's template: begin-of-text, a header naming each role, a blank line (↵↵), the content, an end-of-turn marker.
- Each <|…|> marker is one special token. In a short chat, much of the sequence is the template's own.
- The sequence ends inside an assistant turn, so the most likely continuation is the assistant's reply.
Format: Meta's Llama 3 ChatFormat (github.com/meta-llama/llama3, llama/tokenizer.py, fetched 2026-10-21). No counts shown here; the next page counts them.
The system prompt can count for more only because it comes first, and because post-training may teach the model to favour it.
Model answer
From the template: 9 special markers, 3 role names and 3 blank lines. The text was counted with cl100k_base standing in for Llama 3's tokenizer, so treat it as close, not exact; each marker is exactly one token. Short chats are mostly template. You can't remove those tokens, but every token is billed and prefilled on each call, so make the rest cheap to repeat: static first.
Model answer
Suspect the chat template: its errors "cause silent failures". Diff the token IDs of one fixed prompt before and after; the printed string can't catch it. In the lab's two-message chat, an extra newline keeps 25 tokens but changes token #4; markers tokenized as text give 72 tokens, none special, and print identically; no assistant header gives 21. Fix: use the template that ships with the model.
Model answer
As one layer, not a guarantee. Wallace et al. rank system over user over third-party content; the email is tool output, the lowest of the book's four levels. Misaligned lower-level instructions should be ignored; aligned ones, like "use a formal tone", still followed. It's trained behaviour: on GPT-3.5 it improved defence against system-prompt extraction by 63%, better, not perfect. The template must cooperate too: content encoded as plain text, so a typed <|eot_id|> can't close a turn.
Model answer
Make it specific: the scale and what to do when unsure, "integer scores only" if it returns 4.5, the format with no preamble, and an end marker. A persona changes the standard: in the book's illustration, the same essay "might" get 2 out of 5 plainly and 4 from a first-grade teacher. Add or remove one component at a time and judge the effect. Then cut tokens: in our transcription of the book's Table 5-2, arrow-style examples take 26 tokens against 37.