Senior interview questionAsked at Mistral AI · YouTube
Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
Model answer
Sort failures by stage before changing anything: log the retrieved passages for every bad answer. Right passage missing: fix retrieval with better chunking, hybrid search, query rewriting, metadata filters, or more candidates plus a reranker. Passage present, answer wrong: fix generation by putting it first, trimming distractors, and asking the model to quote and to abstain. Source wrong or stale: fix the data. Then add each failure to the eval set, so the fix stays fixed.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More from Mistral AI interview questions.
More RAG interview questions
- What is retrieval-augmented generation, and how does it reduce hallucination?
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- How do you evaluate a RAG pipeline?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.
- Design an agent for searching internal documents that respects permissions, cites sources and abstains when it should.