Topic 4
Why outputs still break
Even a good prompt leaves failures, and they come in a few recurring kinds:
- Cut off. Generation stops at the output-token limit mid-object. AI Engineering (Ch. 2, "Structured Outputs") warns that otherwise valid JSON "can also be truncated, and thus not parsable". Hands-On Large Language Models (Ch. 6, "Providing Examples") prints a real case: Phi-3-mini's profile stops at
"charisma. - Extra prose. "Sure! Here is the JSON:", a code fence, a friendly closing line. The JSON inside may be fine; the text as a whole doesn't parse.
- Wrong types.
"priority": "2"is a string, not the integer the schema asks for. - Invented labels. Hands-On LLMs (Ch. 6, "Output Verification"): asked for one of two choices, a model "should not come up with a third." It sometimes does.
- Missing or extra fields, and values past a limit, such as a summary that runs too long.
The first two break parsing. The rest parse fine and break the schema: valid JSON with the wrong content.
Predict first
Check outputs against a schema
Deterministic validator · illustrative outputs, not from a model
Ticket (illustrative)
I was charged twice for my March invoice. Please refund the second charge.
{
"category": one of "billing", "bug", "account", "other",
"priority": an integer 1, 2 or 3,
"summary": a string of at most 80 characters
}
No other keys.Parse rate
50% (5/10)
Valid rate
10% (1/10)
Parse rate, no repair
50%
Valid rate, with repair
50%
The outputs are hand-written illustrations, not model outputs, so 50% parsed and 10% valid describe this set, not any model. Click a row to see its error, then edit it: fix the type, invent a new label, delete a quote.
JSON mode doesn't remove these failures. AI Engineering: a JSON mode "typically guarantees only that the outputs are valid JSON—not the content of the JSON objects", and outputs can still be truncated.
Quick check