# How RAG works interview questions > 6 how rag works interview questions for AI and ML engineer roles, each with a model answer. Canonical: https://llmdojo.dev/interview/topic/rag-basics Last updated: 2026-10-23 6 questions: 1 junior, 3 mid-level and 2 senior. Taught in [How RAG Works: Retrieve, Then Generate](https://llmdojo.dev/learn/retrieval-knowledge-systems/rag-architecture). ## [What is retrieval-augmented generation?](https://llmdojo.dev/interview/q/what-is-retrieval-augmented-generation) Level: Junior A **retriever** finds the passages most relevant to a question in your own data, and a **generator** answers with them in its prompt. **Diagram: Retrieval-augmented generation** 1. Search your own documents for passages related to the question, with keywords, embeddings or both. 2. A reranker reads the question with each candidate and keeps the few that actually answer it. 3. The kept passages go into the prompt, with instructions to answer only from them. 4. The model answers from evidence in its context instead of memory alone, so the answer can be checked against its sources. The pattern of Lewis et al. (2020), Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, as used in production systems. It builds context **per query**, so answers can use current, per-user data and **cite** where they came from. ## [Your knowledge base is 150,000 tokens. Do you need RAG?](https://llmdojo.dev/interview/q/your-knowledge-base-is-150-000-tokens-do-you-need-rag) Level: Mid-level Maybe not. Anthropic's guidance: under about **200,000 tokens** you can put the whole knowledge base in the prompt. The trade-off is per-query cost and how well the model uses a long context: sending 150,000 tokens instead of 5 chunks of 300 is **100×** the input on every request. Large or fast-changing data needs retrieval. ## [Which changes force you to re-index, and which don't?](https://llmdojo.dev/interview/q/which-changes-force-you-to-re-index-and-which-don-t) Level: Mid-level The index belongs to the **retriever**. Re-index when the **chunking**, the **embedding model**, or the tokenizer you chunk by changes. Swapping the **generator** doesn't require it: it reads whatever text retrieval hands it. ## [Retrieval finds the right passage 70% of the time. Is a better generator or better retrieval worth more?](https://llmdojo.dev/interview/q/retrieval-finds-the-right-passage-70-of-the-time-is-a-better) Level: Mid-level Usually retrieval. A generator only helps on the questions where it **has the passage**. Illustratively, with 90% accuracy given the passage and 20% without: start at **0.69**; a 95% generator gives **0.725**; 90% retrieval gives **0.83**. With perfect retrieval, the generator's accuracy is the ceiling. ## [How would you decide where a question should be answered from in a RAG system?](https://llmdojo.dev/interview/q/how-would-you-decide-where-a-question-should-be-answered-from-in) Level: Senior Route before retrieving: **Diagram: Deciding where to look** 1. Before searching, decide whether the question needs a search at all, and which source can answer it. 2. Some messages need no retrieval: answering them directly saves a search and keeps irrelevant text out of the prompt. 3. Questions about how things work go to the document index. 4. Questions about numbers in a table (“how many units sold last week?”) need a SQL query written from the schema and executed, not a text search. 5. Whatever the route, the generator answers from what came back. - **No search** for messages like “thanks”: a search always returns something, which then competes for attention. - **Documents** for how things work. - **Tables** for numbers: text-to-SQL from the schema, run it, answer from the result; with many tables, first predict which ones. - **Multi-query** or **multi-hop** when one search can't answer it. A wrong route fails before any ranking happens, so evaluate the router too. ## [When is RAG the wrong tool?](https://llmdojo.dev/interview/q/when-is-rag-the-wrong-tool) Level: Senior When the knowledge fits comfortably in the prompt and rarely changes, a retriever adds work without adding much. When the answer must be **computed**, like a sum over a sales table, text search can't find it: use SQL. And when failures are about **format or behaviour** rather than missing facts, retrieval won't fix them. Long contexts don't make RAG obsolete, though: data outgrows windows, and long contexts aren't always used well. ## Related - [All interview questions](https://llmdojo.dev/interview)