Mid-level interview questionAsked by candidates · YouTube
What is context engineering, and how is it different from prompt engineering?
Model answer
Prompt engineering is wording the instruction. Context engineering decides everything else in the window on each call: retrieved passages, history, tool results, examples, and their order.
Scroll sideways to see the whole diagram
Retrievers propose candidates. BM25 returns only passages that share a word with the query; embedding search always returns a nearest neighbour.
Text version
- Retrievers propose candidates. BM25 returns only passages that share a word with the query; embedding search always returns a nearest neighbour.
- Candidates are ranked by one retriever's score, or by fusing the two rankings (reciprocal rank fusion).
- Selection is a cut: the top k that fit the budget go in. A good passage ranked below the cut is lost, however relevant.
- Then the order. Ranking for a search page and ordering for a model are different jobs: the model reads all of it, but not equally well everywhere.
After AI Engineering Ch. 6, "RAG": "Combining retrieval algorithms" and "Reranking"; Liu et al. 2023 (arXiv:2307.03172), Section 5.
The window is a budget, so the work is selecting what's relevant, compressing what's long, and placing what matters where the model attends best.
Practise the rest
12 Context selection questions, 492 in all, each with a model answer. Free.
More Context selection interview questions
- “Our model has a 200K-token window, so let's just send the whole knowledge base.” What's wrong with that?
- You rank passages with reciprocal rank fusion of BM25 and embeddings, and keep the top one. The passage that answers the query never reaches the model. How?
- Your pipeline concatenates retrieved passages in database order. Does it matter where the most relevant one lands?
- A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
- Your retriever's top passage is usually on topic but often contradicts the right answer. What's going on, and what do you change?
- Design LLM search or RAG for very long documents and very long user queries.