Fallbacks and graceful degradation interview questions
6 questions with model answers
- junior 1
- mid 3
- senior 2
6 fallbacks and graceful degradation questions asked in AI engineer, ML engineer and applied AI engineer interviews, each with a model answer you can say out loud in a minute or two: 1 junior, 3 mid-level and 2 senior. The lesson Fallbacks and Graceful Degradation teaches the ideas behind them.Updated
Model answer
When parts fail or load is too high, return lower-quality, cheaper results instead of nothing: a smaller model, no reranker, a cached answer, the raw passages, or an honest message. Never a hang.
Scroll sideways to see the whole diagram
Normally, the primary model and the full pipeline answer.
Text version
- Normally, the primary model and the full pipeline answer.
- If it's down or rate-limited, a gateway can route to another model. AI Engineering notes API failures are unfortunately common.
- Under load, do less: the SRE book's graceful degradation returns lower-quality, cheaper-to-compute results rather than nothing.
- With no model at all, a cached answer or the raw retrieved passages may still help.
- The bottom rung is an honest message or a person. What's unacceptable is a hang, a blank screen, or an error the user can't act on.
Model answer
If outages are independent, both are down 1% × 0.5% = 0.005% of the time: about 2 minutes a month instead of about 432.
If they share half their outages (same provider or region), it's about 217 minutes. Pick fallbacks by how they fail.
Model answer
Counts failures on a call; past a threshold it opens, failing fast or going straight to the fallback with no timeout wait. After a cool-down it lets one trial through; success closes it.
Scroll sideways to see the whole diagram
Martin Fowler's circuit breaker wraps a remote call and counts its failures.
Text version
- Martin Fowler's circuit breaker wraps a remote call and counts its failures.
- When failures reach a threshold, it trips: further calls return an error immediately, without reaching the failing service, so a fallback can answer at once.
- After a timeout, it lets one trial call through to see whether the problem is fixed.
- If the trial succeeds, the breaker resets; if not, it opens again.
- Fowler's reasons: without it, callers tie up resources waiting on timeouts, failures cascade across systems, and the struggling service keeps getting hit.
In the course's example, users wait 80 s in total instead of 1,000 s during an outage.
Model answer
Queueing everything makes every request slow until most time out: nobody is served. Load shedding rejects some requests fast with a clear “busy”, so the rest are served well. The SRE book's example: return 503 to new requests once too many are in flight.
Model answer
It's a different model: format reliability, refusals and quality may all change, turning an outage into a quality incident. Give the fallback path its own prompt version if needed, its own evaluation run, and metrics split by which model answered.
Model answer
Depends which budget is tighter. Retrying after failure is cheap but can double latency. Parallel requests keep latency near one call but pay for redundant calls every time.
Either way, write the failure behaviour down and exercise it: unused fallback paths rot.