Topic 6

Write the plan down, and wrap-up

Everything in this lesson is a decision best made before it's needed. During an incident, nobody has time to decide which model to fall back to, or what message users should see.

Predict first

Which belongs in a written failure plan for an AI feature?

Only the primary model's nameTimeouts, retry rules, the fallback chain, degraded modes, the overload response and the user-facing messages, plus how each is testedThe training dataNothing: the provider handles failures

Write down the failure behaviourStep 1 / 4

Scroll sideways to see the whole diagram

Each request needs a deadline and a retry rule, from Module 06.

Text version

Each request needs a deadline and a retry rule, from Module 06.
Each model needs a named fallback, with a breaker so an outage doesn't cost every user a timeout.
And the system needs degraded modes and an overload response, decided calmly in advance rather than during an incident.
The SRE book's warning: the code path you never use is the code path that often doesn't work. Exercise fallbacks on purpose.

Wrap-up. From this lesson, you can now:

design a ladder of fallbacks that ends in an honest response, not a hang;
compute what a fallback model buys, and why its failures must be independent of the primary's;
use circuit breakers so an outage doesn't cost every user a timeout;
degrade gracefully under load, and shed load with a fast, clear refusal;
treat fallback models as models to evaluate, and choose between retrying and parallel requests;
hand conversations to people when the system can't serve them well;
write the failure behaviour down, and exercise it before an incident does.

Quick check

One sentence: what does “graceful” mean in graceful degradation?

Nothing ever failsFailures are hidden from usersErrors are loggedWhen parts fail, users still get the best response the remaining parts can give, quickly and honestly, by design rather than by accident

Write the plan down, and wrap-up

Everything in this lesson is a decision best made before it's needed. During an incident, nobody has time to decide which model to fall back to, or what message users should see.

Predict first

Which belongs in a written failure plan for an AI feature?

Write down the failure behaviourStep 1 / 4

Scroll sideways to see the whole diagram

Each request needs a deadline and a retry rule, from Module 06.

Text version

Each request needs a deadline and a retry rule, from Module 06.
Each model needs a named fallback, with a breaker so an outage doesn't cost every user a timeout.
And the system needs degraded modes and an overload response, decided calmly in advance rather than during an incident.
The SRE book's warning: the code path you never use is the code path that often doesn't work. Exercise fallbacks on purpose.

Wrap-up. From this lesson, you can now:

design a ladder of fallbacks that ends in an honest response, not a hang;

compute what a fallback model buys, and why its failures must be independent of the primary's;

use circuit breakers so an outage doesn't cost every user a timeout;

degrade gracefully under load, and shed load with a fast, clear refusal;

treat fallback models as models to evaluate, and choose between retrying and parallel requests;

hand conversations to people when the system can't serve them well;

write the failure behaviour down, and exercise it before an incident does.

Quick check

One sentence: what does “graceful” mean in graceful degradation?