Context window
Also called context windows
The maximum number of tokens a model can process at once: prompt and output together.
Interview questions about Context window
- What is a context window, and why can't GPT-2 small take a 1,025th token even with plenty of memory?
It's the most tokens a model can read at once, shared by prompt, history and output: 1,024 for GPT-2 small.
- A long support conversation no longer fits the context window. Compare the ways to shrink it, and how each one fails.
Every method loses something; you choose what. Dropping the oldest turns is simple, but the book warns it "can be fatally wrong": in an illustrative 40-turn…
- Rapid fire: how do RAG and fine-tuning relate, what do you do when content exceeds the context window, how do you keep conversation state, and how do you manage prompts?
RAG and fine-tuning fill different gaps: RAG supplies changing or citable facts, fine-tuning shapes format, style or a narrow skill.
- What is in-context learning, and what does each example in the prompt cost you?
Learning a task from examples in the prompt, with no change to the weights; each example is a shot.
- Design LLM search or RAG for very long documents and very long user queries.
Treat the context window as a budget. For long documents, chunk along their structure and index chunks plus document summaries, so retrieval goes coarse to…
- What are the components of an LLM agent's memory, what does each one store, and how is information written to and read from it?
Working memory is the context window: task, recent turns, tool results. Session memory keeps the conversation plus a rolling summary.