Topic 3
Asking for a format
When code reads the model's reply, the reply needs a structured output: JSON with known keys, or one label from a fixed list. AI Engineering (Ch. 2, "Structured Outputs") names two cases: tasks that are structured by nature, such as text-to-SQL or classification into valid classes, and tasks whose output feeds another program, as in agent workflows where outputs become tool inputs.
This lesson's running task: turn a support ticket into
{"category": billing | bug | account | other,
"priority": 1, 2 or 3,
"summary": at most 80 characters}
Predict first
Scroll sideways to see the whole diagram
Spell out the keys and their types. If the score must be an integer, say so.
Text version
- Spell out the keys and their types. If the score must be an integer, say so.
- For a classification field, list every allowed label. A label you don't list is one the model may invent.
- An example output removes the remaining ambiguity: key names, quoting, how long a summary is.
- Rule out the extras that break a parser: "Here is the JSON:" and closing pleasantries.
- A prompt makes the format likely, never certain. The next pages check outputs instead of trusting them.
After AI Engineering, Ch. 5, "Specify the output format". Illustrative prompt for the lab's schema.
AI Engineering's advice (Ch. 5, "Specify the output format"): if you want JSON, specify what the keys should be, and give examples if necessary. If the model opens with a preamble such as "Based on the content of this essay…", say explicitly that you don't want one. The same chapter's "Explain, without ambiguity" makes the type point: if you don't want fractional scores, tell the model to output integers.
Hands-On Large Language Models (Ch. 6, "Output Verification", "Providing Examples") shows the effect: asked for a character profile "in JSON format", Phi-3-mini produced a long, truncated object full of keys nobody had specified; given a template with four keys, it returned exactly those four.
Quick check