Mid-level interview question
Two systems each retrieve one relevant result in their top three, at positions 1 and 3. How do rank-aware metrics tell them apart?
Model answer
Average precision averages precision at each relevant position: 1/1 = 1.0 for the first system, 1/3 ≈ 0.33 for the second. MAP averages AP over queries; with one relevant document per query it equals MRR.
nDCG goes further, allowing graded relevance and discounting gains lower down.
Practise the rest
6 RAG evaluation questions, 492 in all, each with a model answer. Free.
More RAG evaluation interview questions
- What are context precision and context recall?
- Why score retrieval and answers separately?
- What's the difference between faithfulness and citation precision?
- Your RAG answers score faithfulness 1.0, yet users report wrong answers. How?
- How do you make a RAG system say “I don't know”, and how do you know it's working?