Chat template
Also called chat templates
The model-specific rule that turns a list of role-tagged messages into the one token sequence the model was fine-tuned on, with special markers around each turn. A wrong template fails silently.
Interview questions about Chat template
- You send a chat API a system message and a user message. What does the model actually receive, and why might the system prompt carry more weight?
One token sequence. The chat template writes each role as marker tokens around its content; AI Engineering: system and user prompts are "processed the same…
- After a library upgrade, your self-hosted Llama 3 gives worse answers and no errors. How do you debug it?
Suspect the chat template: its errors "cause silent failures". Diff the token IDs of one fixed prompt before and after; the printed string can't catch it.
- You're handed fine-tuning code whose loss behaves oddly. What do you check?
Start with what the model actually sees: decode a batch back to text.