Topic 1

Likely is not the same as true

Lessons 1 to 5 followed a prompt all the way to text. Nowhere on that path is there a step that checks whether the text is true. This lesson is about what follows from that.

Try this: step through the replay. At each step the model picks a tokenTokenA piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not characters.Where this is taught because it is likely after the text so far. "Likely" means "this is the kind of thing that comes next in text like this", not "this is correct".

What a hallucination is

HallucinationHallucinationFluent, confident output that is false or not supported by the input.Where this is taught is fluent, confident-sounding output that is false or not supported by any source. Two definitions from the literature:

• AI Engineering (Ch. 2, "The Probabilistic Nature of AI") describes it as a response that is not grounded in facts.
• Ji et al. (2022), a survey of the field, use "content that is nonsensical or unfaithful to the provided source". They split it into intrinsic (contradicts the source) and extrinsic (cannot be checked against the source either way). They also note why it is hard to catch: hallucinated text reads as fluent and natural.

The fluency is the problem. A model that wrote wrong answers in broken grammar would be easy to distrust. A next-token predictor is very good at grammar, because grammar is exactly what makes text likely.