Junior interview question
What is a hallucination, and why can a next-token predictor produce one so fluently?
Model answer
It's fluent, confident output that is false or not grounded in any source. The model was trained to predict the token that actually came next in real text, not the true one, and nothing checks truth. So a wrong answer reads as smoothly as a right one.
Practise the rest
11 Hallucination questions, 492 in all, each with a model answer. Free.
More Hallucination interview questions
- GPT-2 small completes 'The capital of Australia is the city of' with Sydney. Walk me through why a model does that.
- Why do long generated answers leave more room for hallucination than one-word answers?
- You want to automatically flag answers that are probably wrong, using the model's token probabilities. How would you design and validate that?
- You're shipping a support bot that answers from your company's policy documents. What do you put in place to reduce hallucination, and how do you evaluate it?
- Design how a chatbot falls back when it doesn't know the answer, covering uncertainty, retrieval and tools.
- How would you measure an LLM's answer quality and detect hallucinations against the context it was given? Then implement the evaluation component.