State, memory and context interview questions
6 questions with model answers
- junior 1
- mid 3
- senior 2
6 state, memory and context questions asked in AI engineer, ML engineer and applied AI engineer interviews, each with a model answer you can say out loud in a minute or two: 1 junior, 3 mid-level and 2 senior. The lesson State, Memory and Context Over Many Steps teaches the ideas behind them.Updated
Model answer
Internal knowledge (the weights), short-term memory (the context, limited and gone after the task), and long-term memory (external stores reached by retrieval, persistent and editable).
Scroll sideways to see the whole diagram
AI Engineering: the model itself is a memory, of what it was trained on. Information every task needs belongs here, through training or fine-tuning.
Text version
- AI Engineering: the model itself is a memory, of what it was trained on. Information every task needs belongs here, through training or fine-tuning.
- The context is short-term memory: fast to use, limited in size, and it doesn't persist across tasks. It's for what the current step needs.
- External data reached by retrieval is long-term memory, as in a RAG system. Rarely needed information belongs here.
- So the question for each piece of information is how often it's needed, and that decides where it lives.
Place information by how often it's needed.
Model answer
The oldest message is often the task itself. FIFO assumes early messages matter least, and the book warns that can be fatally wrong, especially when they state the purpose.
In the course's 12-step run, dropping the oldest loses the task at step 6 while the context still looks fine.
Model answer
Pin what must stay (task, instructions, constraints) and summarise older steps instead of dropping them: Anthropic calls it compaction.
A summary keeps only what its prompt asks for, so test that later steps still find what they need.
Model answer
Anything that must be exact or must survive: IDs processed, refunds already issued, lists, queues. Structured data stays structured in code; in a context it's just text.
Scroll sideways to see the whole diagram
The context should hold what the next decision needs: the task, a summary, recent results.
Text version
- The context should hold what the next decision needs: the task, a summary, recent results.
- Everything else can live in the harness: a state object, a database, a notes file the agent writes to.
- AI Engineering notes text in a context is unstructured; a memory system that stores structured data, like a spreadsheet of leads or a queue of actions, keeps its structure.
- Anthropic describes keeping lightweight identifiers, such as file paths or stored queries, and loading data with tools at run time, rather than loading everything up front.
Load details just in time with tools, keeping lightweight identifiers in the context.
Model answer
Context rot: as tokens grow, the model's ability to recall information from the context decreases; Anthropic calls the context an attention budget. And every step re-reads, and pays for, the whole context.
So keep it to what the next decision needs, well before the hard limit.
Model answer
The sub-agent knows only its brief. Send the goal, specific inputs, constraints and the shape of the result. Ask for a condensed summary back (Anthropic cites 1,000–2,000 tokens) that includes what it couldn't do or wasn't sure of.
Clean contexts keep each agent focused; the risk is information lost at each boundary.