Topic 3

A chat is one token sequence

A chat templateChat templateThe model-specific rule that turns a list of role-tagged messages into the one token sequence the model was fine-tuned on, with special markers around each turn. A wrong template fails silently.Where this is taught turns the list of messages into the one sequence the model reads. Each model has its own, set by its developers and documented with the model (AI Engineering, Ch. 5, "System PromptSystem promptThe developer's instructions, sent as the first message with the system role. The model gets it in the same sequence as the user's text; it carries more weight only through position and training.Where this is taught and User Prompt").

Llama 3's, from Meta's reference code (ChatFormat in the llama3 repository, fetched 2026-10-21):

1. <|begin_of_text|> once.
2. Per message: <|start_header_id|>, the role, <|end_header_id|>, a blank line, the content, <|eot_id|>.
3. An empty assistant header at the end, so the model's most likely continuation is a reply.

Meta's prompt-format page adds: "Newlines (0x0A) are part of the prompt format". Each <|…|> marker is one special tokenSpecial tokenA vocabulary entry with a control meaning rather than text, such as end-of-text or a chat-role marker. User text must never become one.Where this is taught.

<|begin_of_text|><|start_header_id|>system<|end_header_id|>↵
↵
Translate the text below into French<|eot_id|><|start_header_id|>user<|end_header_id|>↵
↵
How are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>↵
↵

Edit the messages and watch the string and the tokensTokenA piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not characters.Where this is taught. The counts use cl100k_base for text: Llama 3's tokenizerTokenizerThe component that cuts text into tokens and maps each to an integer ID, and turns IDs back into text.Where this is taught splits text with the same pattern but has its own merges, so treat counts as close, not exact. The markers are exact: one token each.

The book's advice holds for any model: "print out the final prompt to double-check if it follows the expected template." This lab is that printout.