Junior interview questionAsked by candidates · YouTube
“Our model has a 200K-token window, so let's just send the whole knowledge base.” What's wrong with that?
Model answer
Fitting isn't helping. Every input token is billed and prefilled on every request, and a long context isn't necessarily used well. In the lesson's help centre, all 14 articles are 299 tokens against 25 for the one that answers, about 12 times the input; with thousands of documents, the ratio grows.
Practise the rest
12 Context selection questions, 492 in all, each with a model answer. Free.
More Context selection interview questions
- You rank passages with reciprocal rank fusion of BM25 and embeddings, and keep the top one. The passage that answers the query never reaches the model. How?
- Your pipeline concatenates retrieved passages in database order. Does it matter where the most relevant one lands?
- A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
- Your retriever's top passage is usually on topic but often contradicts the right answer. What's going on, and what do you change?
- What is context engineering, and how is it different from prompt engineering?
- Design LLM search or RAG for very long documents and very long user queries.