# Examples and structured outputs interview questions

> 10 examples and structured outputs interview questions for AI and ML engineer roles, each with a model answer.

Canonical: https://llmdojo.dev/interview/topic/structured-outputs
Last updated: 2026-10-23

10 questions: 4 junior, 4 mid-level and 2 senior. Taught in [Few-Shot Examples and Structured Outputs](https://llmdojo.dev/learn/prompt-context-engineering/examples-and-structured-outputs).

## [What is in-context learning, and what does each example in the prompt cost you?](https://llmdojo.dev/interview/q/what-is-in-context-learning-and-what-does-each-example-in-the)

Level: Junior

Learning a task from **examples in the prompt**, with **no change to the weights**; each example is a shot. Nothing is learned between calls, so every shot is **input tokens on every request**; the context window caps how many fit. For GPT-4, in Microsoft's 2023 analysis, few-shot gave only limited gains over zero-shot.

## [Your API's JSON mode is on. In what ways can an output still break the code that reads it?](https://llmdojo.dev/interview/q/your-api-s-json-mode-is-on-in-what-ways-can-an)

Level: Junior

JSON mode guarantees **valid JSON, not its content**. Output can still be cut off at the output-token limit, which doesn't parse. And parseable outputs can still carry wrong types (`"2"` for an integer), invented labels, missing or extra fields, or values past a limit. Only a **schema check** catches those.

## [Your few-shot sentiment prompt lists six positive examples, then two negative ones at the very end. What's the risk, and what would you do?](https://llmdojo.dev/interview/q/your-few-shot-sentiment-prompt-lists-six-positive-examples-then-two-negative)

Level: Mid-level

The model may lean towards **the label it saw last**. Zhao et al. traced unstable few-shot accuracy to biases, including towards answers near the end of the prompt; Lu et al. found **order alone** can swing between near state-of-the-art and random-guess performance. Both studied GPT-3-era models, so **test yours**: cover every label and the hard cases, shuffle, compare a few orders on a fixed test set, and end on a marker.

## [You add a repair step that keeps the first JSON object and closes open braces. Nearly every output now parses, but half are still invalid. Why?](https://llmdojo.dev/interview/q/you-add-a-repair-step-that-keeps-the-first-json-object-and)

Level: Mid-level

**Repair fixes syntax, never content**. On the lab's ten hand-written, illustrative outputs, repair lifts parsing from 50% to 90%, but still only 50% are valid: the wrong type, invented label, missing field and long summary were valid JSON all along, and a cut-off summary can't be finished.

**Diagram: Validate, repair, retry**

1. The happy path: the text parses, every field passes its check, and the value goes on to the code that needs it.
2. If parsing fails, a cheap repair may save it: strip the preamble or the code fence, add a missing brace, then check again.
3. If the content is wrong (a missing field, an invented label), repair can't help. Call the model again, ideally telling it what failed.
4. Cap the retries. After the last one, fall back, flag the input for a person and log the bad output: it is your best data on how outputs break.

After AI Engineering, Ch. 2, "Structured Outputs" (post-processing; test time compute: generate until an output fits). Illustrative flow.

Content failures need a **retry with the validator's error**, capped, then a logged fallback.

## [Each attempt gives valid output 90% of the time. A colleague proposes up to three attempts to reach 99.9%. What do you say?](https://llmdojo.dev/interview/q/each-attempt-gives-valid-output-90-of-the-time-a-colleague-proposes)

Level: Senior

The arithmetic is right only if **failures are independent**: 1 − 0.1³ = 99.9% valid, for 1.11 calls on average. That's a best case. The book notes a model "tends to **repeat similar mistakes**", so a retry on the same input may fail the same way. I'd measure **how often retries actually succeed** on our failing inputs, and report parse, valid and correct rates, with and without repair, so a parser fix can't hide a prompt problem.

## [Where in the stack can you push a model towards a format, and what would you use for a classifier that must never invent a label?](https://llmdojo.dev/interview/q/where-in-the-stack-can-you-push-a-model-towards-a-format)

Level: Senior

*AI Engineering* lists five places: prompting, post-processing and test time compute are "more like bandages"; **constrained sampling and finetuning** are the intensive treatment. LinkedIn's defensive parser took correct YAML from 90% to 99.99%, as the book cites, but post-processing works only when mistakes are easy to fix. For fixed labels, constrain sampling to them, or use a classifier head. **Neither makes the label right**: GPT-2 constrained to planet names picks " Earth" over " Jupiter". Still measure the **correct rate** on labelled examples.

## [Zero-shot or few-shot prompting: what's the difference, and when do you add examples?](https://llmdojo.dev/interview/q/zero-shot-or-few-shot-prompting-what-s-the-difference-and-when)

Level: Junior

Zero-shot gives only the instruction; few-shot adds **worked examples** of input and output. Add them when the format or judgement is **hard to describe** but easy to show. Each example costs tokens on every request and the model copies its quirks, so **pick varied examples** and measure that they help.

## [Turn an LLM into a binary classifier using its log-probs and log-sum-exp.](https://llmdojo.dev/interview/q/turn-an-llm-into-a-binary-classifier-using-its-log-probs-and)

Level: Mid-level

Don't parse generated text; read the **next-token distribution** after the prompt.

```python

def p_yes(logits, yes_ids, no_ids):
    # logits: (V,) next-token logits
    lp = logits.log_softmax(-1)
    ly = torch.logsumexp(lp[yes_ids], -1)
    ln = torch.logsumexp(lp[no_ids], -1)
    return torch.sigmoid(ly - ln)
```

Each label has several token spellings, so combine each group with **log-sum-exp**, then renormalise over the two groups: a sigmoid of the difference. Threshold it after **calibrating** on labelled data, since raw probabilities are often skewed, and check token ids, because a leading space changes them.

## [Your LLM writes documents that must follow a strict Word formatting schema: styles, heading levels, tables. How do you make its output conform?](https://llmdojo.dev/interview/q/your-llm-writes-documents-that-must-follow-a-strict-word-formatting-schema)

Level: Mid-level

Don't ask the model for Word markup. Have it emit **structured output**, JSON with only allowed blocks such as heading with level, paragraph, list and table, each naming a permitted style, enforced by schema-constrained decoding.

**Diagram: Constrained sampling on real GPT-2 scores**

1. GPT-2 scores every token in its vocabulary. Left alone, its most likely next token is “ about”, not a planet.
2. Constrained sampling filters the scores before sampling: tokens the format doesn't allow are removed. A grammar decides what is allowed at each step.
3. Only planet names can come out now, and their probabilities are rescaled to sum to 1. The output is always a valid label, but greedy decoding picks “ Earth” (24.8%) over “ Jupiter” (23.7%). The constraint fixes the form, not the content.

Real next-token logits of gpt2 (transformers 5.20.2) for “The largest planet in our solar system is”, from lib/labs/data/sampling.json; probabilities at temperature 1, computed by constrain() in lib/labs/structured.ts.

A **deterministic renderer** then builds the document from the template's styles. Validate rules the schema can't express, like heading order, and retry with the error. Evaluate by diffing rendered styles against the spec.

## [Design an LLM API pipeline that prompts for structured output and validates what comes back.](https://llmdojo.dev/interview/q/design-an-llm-api-pipeline-that-prompts-for-structured-output-and-validates)

Level: Junior

Template the prompt with the **schema** and one example, and use the API's JSON or structured-output mode if it has one. Parse the reply and **validate it** with Pydantic against the same schema. On failure, retry with the validation error, a few times at most, then fail loudly.

**Diagram: Validate, repair, retry**

1. The happy path: the text parses, every field passes its check, and the value goes on to the code that needs it.
2. If parsing fails, a cheap repair may save it: strip the preamble or the code fence, add a missing brace, then check again.
3. If the content is wrong (a missing field, an invented label), repair can't help. Call the model again, ideally telling it what failed.
4. Cap the retries. After the last one, fall back, flag the input for a person and log the bad output: it is your best data on how outputs break.

After AI Engineering, Ch. 2, "Structured Outputs" (post-processing; test time compute: generate until an output fits). Illustrative flow.

## Related
- [All interview questions](https://llmdojo.dev/interview)