Senior interview question
A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
Model answer
Every method loses something; you choose what. Dropping the oldest turns is simple, but the book warns it "can be fatally wrong": in an illustrative 40-turn chat, keeping the last 10 loses turn 1, which account and plan the user is cancelling. A model-written summary keeps more, but costs a call and is new text, so it can omit or misstate details. Truncating, or routing to a larger-window model, are the blunt options. Write down what must survive (goal, IDs, numbers, constraints) and test it early, late and buried.
Practise the rest
12 Context selection questions, 492 in all, each with a model answer. Free.
More Context selection interview questions
- “Our model has a 200K-token window, so let's just send the whole knowledge base.” What's wrong with that?
- You rank passages with reciprocal rank fusion of BM25 and embeddings, and keep the top one. The passage that answers the query never reaches the model. How?
- Your pipeline concatenates retrieved passages in database order. Does it matter where the most relevant one lands?
- Your retriever's top passage is usually on topic but often contradicts the right answer. What's going on, and what do you change?
- What is context engineering, and how is it different from prompt engineering?
- Design LLM search or RAG for very long documents and very long user queries.