# Caching repeated work interview questions > 6 caching repeated work interview questions for AI and ML engineer roles, each with a model answer. Canonical: https://llmdojo.dev/interview/topic/caching Last updated: 2026-10-23 6 questions: 1 junior, 3 mid-level and 2 senior. Taught in [Caching Repeated Work: Prompts, Prefixes and Answers](https://llmdojo.dev/learn/inference-optimization/caching-repeated-work). ## [Name three caches that reuse work when serving an LLM, and the scope of each.](https://llmdojo.dev/interview/q/name-three-caches-that-reuse-work-when-serving-an-llm-and-the) Level: Junior The KV cache works **within one request**, so each decode step computes only the new token's keys and values: after a 960-token prompt on our laptop CPU, 9.6 ms a step, against 228.8 ms without. A prompt cache keeps a shared opening's keys and values **across requests**. An answer cache **skips the model entirely**. ## [Someone added today's date as the first line of every prompt, and the prompt-cache hit rate dropped to zero. Why, and how should prompts be laid out?](https://llmdojo.dev/interview/q/someone-added-today-s-date-as-the-first-line-of-every-prompt) Level: Mid-level A prompt cache reuses only an **identical prefix**, and the match stops at the first differing token: here, the very first. It isn't a provider quirk: a token's keys and values depend on every token before it. **Diagram: Where a prefix match breaks** 1. Put what never changes first. The cache can reuse everything up to the first token that differs. 2. Put something that changes at the top, like a date, and the match ends at token one: nothing is reused. 3. This is not a provider quirk. Through attention, each position's keys and values depend on everything before it, so only an identical prefix gives identical values. Matching rule: Anthropic and OpenAI prompt-caching docs (fetched 2026-10-21). Illustrative prompts. Order prompts **static first**: fixed instructions and tools, then shared reference material, then per-request parts like the date, retrieved snippets and the question. ## [When does a prompt cache make the input bill higher rather than lower?](https://llmdojo.dev/interview/q/when-does-a-prompt-cache-make-the-input-bill-higher-rather-than) Level: Mid-level When **writes aren't followed by reads**. Anthropic charges 1.25 times the base input price for a 5-minute cache write and 0.1 times for a read (as of 2026-10-21). A 2,000-token prefix sent once costs 24% more than not caching. Rare traffic is the same: the entry expires, so **every request pays to write** it again. Shared by 10 requests, a 10,000-token prefix saves 77%. ## [A code assistant sends a 2,000-token system prompt and writes long answers. Users complain answers take too long to finish. Will a prefix cache fix it?](https://llmdojo.dev/interview/q/a-code-assistant-sends-a-2-000-token-system-prompt-and-writes) Level: Senior **Mostly not**. A prefix cache removes the prefix's prefill, so it **cuts TTFT** and input cost, **not decode**. In our CPU estimate with 128 shared and 64 unique tokens and 400 output tokens, the first token came about 48% sooner but the reply finished only about 1% sooner: 3.32 s to 3.30 s. That estimate is optimistic, too: it ignores attention to the cached prefix and loading the cache. To finish sooner, change decode: shorter outputs, faster hardware, or fewer bytes per weight. ## [Product wants a semantic answer cache in front of your support bot to cut cost. What risks would you raise, and what would you measure first?](https://llmdojo.dev/interview/q/product-wants-a-semantic-answer-cache-in-front-of-your-support-bot) Level: Senior An answer cache serves a reply written for an earlier request. Risks: stale answers to time-sensitive queries; leaks, like the book's return-policy answer built from user X's details and served to user Y; and, with semantic matching, **wrong hits**. Embeddings capture **topic more than truth**: with all-MiniLM-L6-v2, "The Earth orbits the Sun." and its reverse scored 0.982, while the best true paraphrase scored 0.645. I'd measure hit rate and **wrong-hit rate** on our own queries, and keep personal queries out. ## [What is system-prompt caching, and what does it actually save?](https://llmdojo.dev/interview/q/what-is-system-prompt-caching-and-what-does-it-actually-save) Level: Mid-level The provider keeps the **processed prefix**, the keys and values prefill computed for your system prompt, and reuses them when the next request starts with the same tokens. You skip that prefill: **cheaper input and a faster first token**. **Diagram: Where a prefix match breaks** 1. Put what never changes first. The cache can reuse everything up to the first token that differs. 2. Put something that changes at the top, like a date, and the match ends at token one: nothing is reused. 3. This is not a provider quirk. Through attention, each position's keys and values depend on everything before it, so only an identical prefix gives identical values. Matching rule: Anthropic and OpenAI prompt-caching docs (fetched 2026-10-21). Illustrative prompts. It only matches an **exact prefix**, so put static text first and anything that changes, like dates or user data, after it. ## Related - [All interview questions](https://llmdojo.dev/interview)