Mid-level interview questionAsked by candidates · YouTube
How do you evaluate a RAG pipeline?
Model answer
Score the two halves separately. Retrieval: on questions with known source passages, does the right one land in the top k? Generation: is the answer faithful to the retrieved passages, and does it answer the question? Use a judge model checked against human labels, on a fixed set run after every change. Bad retrieval and bad generation need different fixes.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More RAG interview questions
- What is retrieval-augmented generation, and how does it reduce hallucination?
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.
- Design an agent for searching internal documents that respects permissions, cites sources and abstains when it should.