Mid-level interview questionAsked at Microsoft · interview write-up
Your LLM writes documents that must follow a strict Word formatting schema: styles, heading levels, tables. How do you make its output conform?
Model answer
Don't ask the model for Word markup. Have it emit structured output, JSON with only allowed blocks such as heading with level, paragraph, list and table, each naming a permitted style, enforced by schema-constrained decoding.
Scroll sideways to see the whole diagram
GPT-2 scores every token in its vocabulary. Left alone, its most likely next token is “ about”, not a planet.
Text version
- GPT-2 scores every token in its vocabulary. Left alone, its most likely next token is “ about”, not a planet.
- Constrained sampling filters the scores before sampling: tokens the format doesn't allow are removed. A grammar decides what is allowed at each step.
- Only planet names can come out now, and their probabilities are rescaled to sum to 1. The output is always a valid label, but greedy decoding picks “ Earth” (24.8%) over “ Jupiter” (23.7%). The constraint fixes the form, not the content.
Real next-token logits of gpt2 (transformers 5.20.2) for “The largest planet in our solar system is”, from lib/labs/data/sampling.json; probabilities at temperature 1, computed by constrain() in lib/labs/structured.ts.
A deterministic renderer then builds the document from the template's styles. Validate rules the schema can't express, like heading order, and retry with the error. Evaluate by diffing rendered styles against the spec.
Practise the rest
10 Structured outputs questions, 492 in all, each with a model answer. Free.
More from Microsoft interview questions.
More Structured outputs interview questions
- What is in-context learning, and what does each example in the prompt cost you?
- Your API's JSON mode is on. In what ways can an output still break the code that reads it?
- Your few-shot sentiment prompt lists six positive examples, then two negative ones at the very end. What's the risk, and what would you do?
- You add a repair step that keeps the first JSON object and closes open braces. Nearly every output now parses, but half are still invalid. Why?
- Each attempt gives valid output 90% of the time. A colleague proposes up to three attempts to reach 99.9%. What do you say?
- Where in the stack can you push a model towards a format, and what would you use for a classifier that must never invent a label?