# Budgets, rate limits and usage controls interview questions > 6 budgets, rate limits and usage controls interview questions for AI and ML engineer roles, each with a model answer. Canonical: https://llmdojo.dev/interview/topic/budgets Last updated: 2026-10-23 6 questions: 1 junior, 3 mid-level and 2 senior. Taught in [Budgets, Rate Limits and Usage Controls](https://llmdojo.dev/learn/production-ai-systems/budgets-and-usage-controls). ## [What three budgets does an AI feature have?](https://llmdojo.dev/interview/q/what-three-budgets-does-an-ai-feature-have) Level: Junior **Cost**, **rate** and **latency**. Each is a requirement enforced in code with a planned behaviour when it's hit. A provider's spend cap stops every feature on the account until it resets. **Diagram: Three budgets every AI feature has** 1. Cost: what each request, each user and the whole feature may spend. Module 02 showed how to compute it. 2. Rate: how many requests and tokens per minute, both the limits a provider imposes on you and the ones you impose on your users. 3. Latency: how long a user waits, for the first token and for the whole answer. 4. A budget that's only a number in a spreadsheet isn't a control. The gateway or the application enforces it. 5. Hitting a budget is a failure mode like an outage, and needs a planned response. ## [Your feature costs too much. Which levers do you compare?](https://llmdojo.dev/interview/q/your-feature-costs-too-much-which-levers-do-you-compare) Level: Mid-level Prompt caching of a shared prefix, routing easy requests to a smaller model, shorter outputs, and fewer input tokens. In the course's illustrative model, caching or routing roughly halves a $27,000 month; both together reach about $8,000. Each has a quality side: measure it before shipping. ## [When is caching whole answers dangerous?](https://llmdojo.dev/interview/q/when-is-caching-whole-answers-dangerous) Level: Mid-level When the answer depends on **who asked**. The book's example: a return-policy answer built from user X's membership, cached as generic, then served to user Y. **Diagram: What to cache, and what never to** 1. AI Engineering: exact caching suits answers that are the same for everyone, and is especially appealing for multi-step or slow work like retrieval, SQL or web search. 2. User-specific queries (“What's the status of my recent order?”) are unlikely to be reused, and time-sensitive ones go stale. Many teams train a classifier to decide what to cache. 3. The book's warning: a return-policy question looks generic, but the answer used user X's membership details. Cached as generic, it reveals X's information to the next user who asks. 4. So a cache key must include everything the answer depended on, or the answer must not be cached. Key the cache on everything the answer depended on, skip user-specific and time-sensitive queries, and invalidate on prompt or data changes. ## [You're under 60 requests a minute but still get 429s. Why?](https://llmdojo.dev/interview/q/you-re-under-60-requests-a-minute-but-still-get-429s-why) Level: Mid-level Limits use a **token bucket**: capacity refills continuously, so short bursts can be rejected even when the per-minute total is fine. Smooth bursts, wait at least `retry-after` with backoff, separate batch from live traffic, and use prompt caching, which for most Claude models doesn't count toward input-token limits. ## [How would you structure usage limits across an organisation?](https://llmdojo.dev/interview/q/how-would-you-structure-usage-limits-across-an-organisation) Level: Senior Nested: provider limits for the organisation; per-application limits in the gateway so one can't starve others; **per-user** limits against abuse and compromised accounts; **per-request** caps on tokens and agent steps. Track spend daily, project the month, and act before the hard cap. ## [Your answers must start within one second. How do you budget it?](https://llmdojo.dev/interview/q/your-answers-must-start-within-one-second-how-do-you-budget-it) Level: Senior Add up the stages before the first token: router, retrieval, reranker, the model's time to first token. Streaming hides generation, but not for answers needed whole, like JSON or output-guardrail checks. Parallelise independent stages, skip optional ones under load, and budget **percentiles**, not averages. ## Related - [All interview questions](https://llmdojo.dev/interview)