Topic 2

Choosing and ordering examples

Examples say what you want more clearly than instructions do, so they also say the wrong thing clearly. Which examples you pick, how you write them and what order they come in all change the answer.

Predict first

You give a sentiment classifier eight examples: six positive, then two negative at the very end. What risk does the research below point to?

None: models ignore example orderThe model will refuse to answerThe prompt becomes invalid JSONThe model may lean towards the label it saw last

What to check in a set of examplesStep 1 / 4

Scroll sideways to see the whole diagram

A compact few-shot block: the same short format for every example, then the real input in that format.

Text version

A compact few-shot block: the same short format for every example, then the real input in that format.
Choose examples that cover the labels and the inputs you see in production. Examples are the clearest statement of what you want.
Order matters. Zhao et al. (2021) found models lean towards answers placed near the end of the prompt; Lu et al. (2021) found order alone can swing a model between near state of the art and random guessing.
End the input with the same marker the examples use, so the model knows the label comes next.

After AI Engineering, Ch. 5, "Provide examples" and "Specify the output format"; Zhao et al. (2021); Lu et al. (2021). Illustrative examples.

Practical rules that follow:

Cover the space. Include every label and the hard, borderline inputs you actually see, with labels roughly balanced.
Shuffle, then test. Order is a variable: compare a few orders on a fixed test set before trusting one.
Keep the format compact. AI Engineering (Ch. 5, "Provide examples", Table 5-2) shows the same edible/inedible examples written as "chickpea --> edible" taking 27 GPT-4 tokens instead of 38 with "Input:/Output:" lines. If both work equally well, prefer the shorter one.
End on a marker. Without one the model may keep writing the input. In the book's Table 5-3, a prompt ending "chicken" gets "tacos --> edible" appended; ending it "chicken -->" gets "edible".

Both papers (fetched 2026-10-21) studied GPT-3-era models; check your own model rather than assuming it is as sensitive.

Quick check

Why end a few-shot classification prompt with 'new input -->' rather than just 'new input'?

It saves tokensModels can't read the last line otherwiseThe marker tells the model the label comes next, so it answers instead of continuing the listIt forces JSON output

Choosing and ordering examples

Examples say what you want more clearly than instructions do, so they also say the wrong thing clearly. Which examples you pick, how you write them and what order they come in all change the answer.

Predict first

You give a sentiment classifier eight examples: six positive, then two negative at the very end. What risk does the research below point to?

None: models ignore example orderThe model will refuse to answerThe prompt becomes invalid JSONThe model may lean towards the label it saw last

What to check in a set of examplesStep 1 / 4

Scroll sideways to see the whole diagram

A compact few-shot block: the same short format for every example, then the real input in that format.

Text version

A compact few-shot block: the same short format for every example, then the real input in that format.
Choose examples that cover the labels and the inputs you see in production. Examples are the clearest statement of what you want.
Order matters. Zhao et al. (2021) found models lean towards answers placed near the end of the prompt; Lu et al. (2021) found order alone can swing a model between near state of the art and random guessing.
End the input with the same marker the examples use, so the model knows the label comes next.

After AI Engineering, Ch. 5, "Provide examples" and "Specify the output format"; Zhao et al. (2021); Lu et al. (2021). Illustrative examples.

Practical rules that follow:

Cover the space. Include every label and the hard, borderline inputs you actually see, with labels roughly balanced.

Shuffle, then test. Order is a variable: compare a few orders on a fixed test set before trusting one.

Keep the format compact. AI Engineering (Ch. 5, "Provide examples", Table 5-2) shows the same edible/inedible examples written as "chickpea --> edible" taking 27 GPT-4 tokens instead of 38 with "Input:/Output:" lines. If both work equally well, prefer the shorter one.

End on a marker. Without one the model may keep writing the input. In the book's Table 5-3, a prompt ending "chicken" gets "tacos --> edible" appended; ending it "chicken -->" gets "edible".

Both papers (fetched 2026-10-21) studied GPT-3-era models; check your own model rather than assuming it is as sensitive.

Quick check

Why end a few-shot classification prompt with 'new input -->' rather than just 'new input'?

It saves tokensModels can't read the last line otherwiseThe marker tells the model the label comes next, so it answers instead of continuing the listIt forces JSON output