Mid-level interview question
Your knowledge base is 150,000 tokens. Do you need RAG?
Model answer
Maybe not. Anthropic's guidance: under about 200,000 tokens you can put the whole knowledge base in the prompt.
The trade-off is per-query cost and how well the model uses a long context: sending 150,000 tokens instead of 5 chunks of 300 is 100× the input on every request. Large or fast-changing data needs retrieval.
Practise the rest
6 RAG basics questions, 492 in all, each with a model answer. Free.
More RAG basics interview questions
- What is retrieval-augmented generation?
- Which changes force you to re-index, and which don't?
- Retrieval finds the right passage 70% of the time. Is a better generator or better retrieval worth more?
- How would you decide where a question should be answered from in a RAG system?
- When is RAG the wrong tool?