Mid-level interview question

You're under 60 requests a minute but still get 429s. Why?

Q: You're under 60 requests a minute but still get 429s. Why?

Limits use a token bucket: capacity refills continuously, so short bursts can be rejected even when the per-minute total is fine. Smooth bursts, wait at least retry-after with backoff, separate batch from live traffic, and use prompt caching, which for most Claude models doesn't count toward input-token limits.

Model answer

Limits use a token bucket: capacity refills continuously, so short bursts can be rejected even when the per-minute total is fine. Smooth bursts, wait at least retry-after with backoff, separate batch from live traffic, and use prompt caching, which for most Claude models doesn't count toward input-token limits.

Practise the rest

6 Budgets questions, 492 in all, each with a model answer. Free.

Read the lesson Practise Budgets

What three budgets does an AI feature have?
Your feature costs too much. Which levers do you compare?
When is caching whole answers dangerous?
How would you structure usage limits across an organisation?
Your answers must start within one second. How do you budget it?

Model answer

More Budgets interview questions

Model answer

More Budgets interview questions