# Chunking and indexing interview questions > 6 chunking and indexing interview questions for AI and ML engineer roles, each with a model answer. Canonical: https://llmdojo.dev/interview/topic/chunking Last updated: 2026-10-23 6 questions: 1 junior, 3 mid-level and 2 senior. Taught in [Chunking and Indexing](https://llmdojo.dev/learn/retrieval-knowledge-systems/chunking-and-indexing). ## [Why do RAG systems chunk documents?](https://llmdojo.dev/interview/q/why-do-rag-systems-chunk-documents) Level: Junior Embedding models and generators have input limits, and one vector for a whole document **blurs every topic** in it. Chunks give each idea its own vector, so a specific question finds the specific passage, and only that passage spends prompt tokens. ## [How much overlap should chunks have?](https://llmdojo.dev/interview/q/how-much-overlap-should-chunks-have) Level: Mid-level Enough to keep whole what must stay whole. A span of L words always lands inside some chunk when **overlap ≥ L − 1**: for a 9-word answer, overlap 8. The cost: more chunks to embed and store (13 instead of 7 in the course's example), and repeated text when neighbours are both retrieved. ## [Fixed-size or recursive chunking?](https://llmdojo.dev/interview/q/fixed-size-or-recursive-chunking) Level: Mid-level Recursive, usually: split by section, then paragraph, then sentence, until each piece fits. Cuts land where the author put them. Fixed sizes ignore meaning, so whether a fact survives is luck. Some documents suggest their own unit: a question-and-answer pair, or a code-aware splitter. ## [A chunk is whole and correct but never retrieved. What's the fix?](https://llmdojo.dev/interview/q/a-chunk-is-whole-and-correct-but-never-retrieved-what-s-the) Level: Mid-level Give it its context. Cut from its document, it may never say what it's about: “It usually arrives in 5 to 7 business days” doesn't say “refund”. **Diagram: Adding context to a chunk before indexing** 1. Cut out of its document, a chunk can lose what it's about. This one never says “refund”. 2. The cheap fix is metadata: the document title, the section heading, tags, or entities such as error codes. 3. Anthropic's contextual retrieval uses a model instead: given the whole document and the chunk, it writes a short context, usually 50–100 tokens, situating the chunk. 4. The context is prepended to the chunk. 5. Then the augmented chunk is indexed, so a search for “refund” can now find it. Prepend the title and section, add keywords or entities, list questions it answers, or generate a short context with a model, as Anthropic's contextual retrieval does. ## [How do you choose a chunk size?](https://llmdojo.dev/interview/q/how-do-you-choose-a-chunk-size) Level: Senior By experiment. Smaller chunks let more distinct pieces fit (halve the size, fit twice as many) but split information and double the vectors to compute, store and search. Larger ones keep context together but carry more irrelevant text. Both are bounded by the embedding model's and generator's limits. **Diagram: Smaller chunks or larger?** 1. Smaller chunks mean more of them fit in the prompt, giving the model a wider range of information. 2. But information that spans a boundary can be lost: if only one half is retrieved, the model never sees the other. 3. And computational overhead grows: AI Engineering notes that halving the chunk size doubles the chunks to index and the vectors to generate and store, and the search space. 4. Larger chunks keep context together, but each one spends more of the budget, much of it on text the question doesn't need. 5. A chunk can't exceed what the embedding model or the generator can read. Between those limits there's no universal best: experiment on your own queries. The book says there's no universal best: measure retrieval on real questions with known answers at a few sizes. ## [Anthropic reports large gains from contextual retrieval. What exactly did they measure, and what does it cost?](https://llmdojo.dev/interview/q/anthropic-reports-large-gains-from-contextual-retrieval-what-exactly-did-they-measure) Level: Senior **1 − recall@20**: the share of relevant chunks missing from the top 20. Contextual embeddings cut it **35%** (5.7% → 3.7%); adding contextual BM25, **49%**; adding a reranker, **67%** (to 1.9%), on their evaluation sets. The cost is a model call per chunk at indexing time, repeated whenever documents change, plus longer chunks to embed and store. ## Related - [All interview questions](https://llmdojo.dev/interview)