Junior interview question
What is retrieval-augmented generation?
Model answer
A retriever finds the passages most relevant to a question in your own data, and a generator answers with them in its prompt.
Scroll sideways to see the whole diagram
Search your own documents for passages related to the question, with keywords, embeddings or both.
Text version
- Search your own documents for passages related to the question, with keywords, embeddings or both.
- A reranker reads the question with each candidate and keeps the few that actually answer it.
- The kept passages go into the prompt, with instructions to answer only from them.
- The model answers from evidence in its context instead of memory alone, so the answer can be checked against its sources.
The pattern of Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, as used in production systems.
It builds context per query, so answers can use current, per-user data and cite where they came from.
Practise the rest
6 RAG basics questions, 492 in all, each with a model answer. Free.
More RAG basics interview questions
- Your knowledge base is 150,000 tokens. Do you need RAG?
- Which changes force you to re-index, and which don't?
- Retrieval finds the right passage 70% of the time. Is a better generator or better retrieval worth more?
- How would you decide where a question should be answered from in a RAG system?
- When is RAG the wrong tool?