Mid-level interview questionAsked at Amazon · interview write-up
Design a production RAG system end to end, from ingesting documents to serving answers.
Model answer
Offline: parse documents, chunk on their structure, embed, and index chunks with metadata and permissions in a vector and keyword index, updated incrementally.
Scroll sideways to see the whole diagram
Search your own documents for passages related to the question, with keywords, embeddings or both.
Text version
- Search your own documents for passages related to the question, with keywords, embeddings or both.
- A reranker reads the question with each candidate and keeps the few that actually answer it.
- The kept passages go into the prompt, with instructions to answer only from them.
- The model answers from evidence in its context instead of memory alone, so the answer can be checked against its sources.
The pattern of Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, as used in production systems.
Online: rewrite the query, run hybrid search filtered by the user's access, rerank, and pack the best passages into a prompt that says to cite them and abstain when the answer isn't there.
Around it: an eval set scoring retrieval and faithfulness separately, logged passages, caching, and a per-stage latency budget.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More from Amazon interview questions.
More RAG interview questions
- What is retrieval-augmented generation, and how does it reduce hallucination?
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- How do you evaluate a RAG pipeline?
- Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.