Junior interview question
What is in-context learning, and what does each example in the prompt cost you?
Model answer
Learning a task from examples in the prompt, with no change to the weights; each example is a shot. Nothing is learned between calls, so every shot is input tokens on every request; the context window caps how many fit. For GPT-4, in Microsoft's 2023 analysis, few-shot gave only limited gains over zero-shot.
Practise the rest
10 Structured outputs questions, 492 in all, each with a model answer. Free.
More Structured outputs interview questions
- Your API's JSON mode is on. In what ways can an output still break the code that reads it?
- Your few-shot sentiment prompt lists six positive examples, then two negative ones at the very end. What's the risk, and what would you do?
- You add a repair step that keeps the first JSON object and closes open braces. Nearly every output now parses, but half are still invalid. Why?
- Each attempt gives valid output 90% of the time. A colleague proposes up to three attempts to reach 99.9%. What do you say?
- Where in the stack can you push a model towards a format, and what would you use for a classifier that must never invent a label?
- Zero-shot or few-shot prompting: what's the difference, and when do you add examples?