Junior interview questionAsked at Scale AI · YouTube
What is retrieval-augmented generation, and how does it reduce hallucination?
Model answer
Before the model answers, retrieve relevant passages from your documents and put them in the prompt, so it answers from evidence in its context, not memory alone.
Scroll sideways to see the whole diagram
Search your own documents for passages related to the question, with keywords, embeddings or both.
Text version
- Search your own documents for passages related to the question, with keywords, embeddings or both.
- A reranker reads the question with each candidate and keeps the few that actually answer it.
- The kept passages go into the prompt, with instructions to answer only from them.
- The model answers from evidence in its context instead of memory alone, so the answer can be checked against its sources.
The pattern of Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, as used in production systems.
It cuts hallucination when retrieval finds the right passage and the prompt says to answer only from it, cite it, and admit when it isn't there.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More from Scale AI interview questions.
More RAG interview questions
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- How do you evaluate a RAG pipeline?
- Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.
- Design an agent for searching internal documents that respects permissions, cites sources and abstains when it should.