Senior interview question
Your retriever's top passage is usually on topic but often contradicts the right answer. What's going on, and what do you change?
Model answer
Retrieval scores similarity, not whether a passage answers. For "is this toy safe for a two-year-old", BM25, embeddings and their fusion all put "safe for children over three" above "not safe for children under three", the one that answers; embeddings barely register "not". With top 1 the model gets only the misleading passage. Evaluate retrieval against questions with known answers, keep more than one candidate, and rerank with a more precise second stage.
Practise the rest
12 Context selection questions, 492 in all, each with a model answer. Free.
More Context selection interview questions
- “Our model has a 200K-token window, so let's just send the whole knowledge base.” What's wrong with that?
- You rank passages with reciprocal rank fusion of BM25 and embeddings, and keep the top one. The passage that answers the query never reaches the model. How?
- Your pipeline concatenates retrieved passages in database order. Does it matter where the most relevant one lands?
- A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
- What is context engineering, and how is it different from prompt engineering?
- Design LLM search or RAG for very long documents and very long user queries.