Mid-level interview question
What's the difference between faithfulness and citation precision?
Model answer
Faithfulness (Ragas): claims supported by any retrieved passage ÷ all claims. Citation precision: citations that support the sentence they're attached to.
A true sentence citing the wrong passage is faithful but fails citation precision. Liu et al. found only 74.5% of generative search engines' citations supported their sentence.
Practise the rest
6 RAG evaluation questions, 492 in all, each with a model answer. Free.
More RAG evaluation interview questions
- What are context precision and context recall?
- Two systems each retrieve one relevant result in their top three, at positions 1 and 3. How do rank-aware metrics tell them apart?
- Why score retrieval and answers separately?
- Your RAG answers score faithfulness 1.0, yet users report wrong answers. How?
- How do you make a RAG system say “I don't know”, and how do you know it's working?