Senior interview questionAsked at Microsoft · interview write-up
Design per-user memory so a chatbot can personalise its answers over time.
Model answer
Keep two stores per user: a small profile of stable facts and preferences, always in the prompt, and episode summaries retrieved by relevance.
Scroll sideways to see the whole diagram
The context window is finite: the prompt and the reply share it.
Text version
- The context window is finite: the prompt and the reply share it.
- Part of the prompt is fixed. The rest, such as retrieved passages, is chosen for each request, and that is where the choices are.
- In this lesson's small help centre, all 14 articles come to 299 tokens; the article that answers "how do I stop paying every month" is 25. About 12 times the input for the same answer.
- Input tokens are billed and prefilled on every request. Spend them on passages that help.
Token counts: o200k_base on the 14 help articles of lib/labs/data/search-lab.json (lib/labs/context.test.ts). Cost and latency: AI Engineering Ch. 6, "RAG".
After each conversation, a model extracts atomic facts with a timestamp and source; newer facts override conflicting ones, and duplicates merge. At query time, fit the profile plus the top retrieved memories into a fixed token budget.
The hard part is trust: users can see and delete memories, sensitive data needs consent and retention limits, stores are isolated per user, and remembered text is data, never instructions. Measure whether it helps.
Practise the rest
12 Context selection questions, 492 in all, each with a model answer. Free.
More from Microsoft interview questions.
More Context selection interview questions
- “Our model has a 200K-token window, so let's just send the whole knowledge base.” What's wrong with that?
- You rank passages with reciprocal rank fusion of BM25 and embeddings, and keep the top one. The passage that answers the query never reaches the model. How?
- Your pipeline concatenates retrieved passages in database order. Does it matter where the most relevant one lands?
- A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
- Your retriever's top passage is usually on topic but often contradicts the right answer. What's going on, and what do you change?
- What is context engineering, and how is it different from prompt engineering?