Topic 1
Learning from examples in the prompt
A model can pick up a task from examples in the prompt, with no change to its weights. AI Engineering (Ch. 5, "In-Context Learning: Zero-Shot and Few-Shot") calls this in-context learning, a term from the GPT-3 paper (Brown et al., 2020). GPT-3 was trained only to predict the next token, yet it could translate or answer SAT questions from examples in its context.
Each example is a shot. Five examples make a 5-shot prompt; none makes it zero-shot.
Scroll sideways to see the whole diagram
Zero-shot: a task description and the task, no examples. The model has to infer the format and the labels from the words alone.
Text version
- Zero-shot: a task description and the task, no examples. The model has to infer the format and the labels from the words alone.
- Few-shot: add worked examples, each called a shot. Two examples make it 2-shot. The weights don't change; the examples are just more prompt.
- Every shot costs input tokens on every call, and the context window caps how many fit.
After AI Engineering, Ch. 5, "In-Context Learning: Zero-Shot and Few-Shot". Illustrative prompt.
Predict first
The book's reading: stronger models follow instructions better, so they need fewer examples. It adds a caveat: the study may undercount domain-specific cases. If a model saw little of an API in training (its example is the Ibis dataframe API), examples in the prompt can still make a big difference.
How many shots? The book says it depends on the model and the application: experiment. More examples usually help, but each one lengthens every prompt, raising cost, and the context window caps how many fit.
Quick check