Junior interview question
What kinds of memory does an AI agent have?
Model answer
Internal knowledge (the weights), short-term memory (the context, limited and gone after the task), and long-term memory (external stores reached by retrieval, persistent and editable).
Scroll sideways to see the whole diagram
AI Engineering: the model itself is a memory, of what it was trained on. Information every task needs belongs here, through training or fine-tuning.
Text version
- AI Engineering: the model itself is a memory, of what it was trained on. Information every task needs belongs here, through training or fine-tuning.
- The context is short-term memory: fast to use, limited in size, and it doesn't persist across tasks. It's for what the current step needs.
- External data reached by retrieval is long-term memory, as in a RAG system. Rarely needed information belongs here.
- So the question for each piece of information is how often it's needed, and that decides where it lives.
Place information by how often it's needed.
Practise the rest
6 Agent memory questions, 492 in all, each with a model answer. Free.
More Agent memory interview questions
- What goes wrong if the harness just drops the oldest messages when the context fills?
- How do you keep a long run within its context without losing what matters?
- What belongs in harness state rather than the context?
- Your context fits the window. Why might you still manage it?
- How would you design a handoff to a sub-agent?