Lesson 4 · 7 topics · 24 min · intermediate

When Steps Fail: Retries, Timeouts, Idempotency and Checkpoints

You will be able to

Sort tool failures into transient, overload and permanent, and route each to a retry, a backoff, or the model.
Set timeouts from a latency percentile, and treat a timed-out write as an unknown outcome.
Retry with exponential backoff and jitter, and explain what jitter buys.
Prevent retry amplification: retry at one layer, cap retries per request, keep a retry budget, and count the model's own retries.
Make writes safe to retry with idempotency keys generated by the harness.
Checkpoint after every step so crashes and pauses resume without repeating side effects.
Stop and escalate after repeated failures.

Topics

Your review

Sign in to keep your quick-check answers. The ones you miss come back for review after 1, 3 and 7 days, until they stick.

Sources

AI Engineering · Chip Huyen, 2025
Chip Huyen. O'Reilly. Used: Ch. 6 "RAG and Agents": "Agent Failure Modes and Evaluation" > "Tool failures".
Site Reliability Engineering · Betsy Beyer et al., 2016
Betsy Beyer, Chris Jones, Jennifer Petoff & Niall Richard Murphy (eds.). O'Reilly. Used: Ch. 22 "Addressing Cascading Failures": "Retries", "Latency and Deadlines" (fetched 2026-10-23). Open source
Timeouts, retries, and backoff with jitter · Marc Brooker (Amazon Builders' Library), 2019
Marc Brooker (Amazon Builders' Library). Used: Timeouts; Retries and backoff; Jitter; Idempotent APIs (fetched 2026-10-23). Open source
Exponential Backoff And Jitter · Marc Brooker (AWS Architecture Blog), 2015
Marc Brooker (AWS Architecture Blog). Used: Conclusions (fetched 2026-10-23). Open source
Idempotent requests (Stripe API reference) · Stripe
Stripe. Used: API v1 behaviour (fetched 2026-10-23). Open source

You will be able to

Sort tool failures into transient, overload and permanent, and route each to a retry, a backoff, or the model.

Set timeouts from a latency percentile, and treat a timed-out write as an unknown outcome.

Retry with exponential backoff and jitter, and explain what jitter buys.

Prevent retry amplification: retry at one layer, cap retries per request, keep a retry budget, and count the model's own retries.

Make writes safe to retry with idempotency keys generated by the harness.

Checkpoint after every step so crashes and pauses resume without repeating side effects.

Stop and escalate after repeated failures.

Sources

AI Engineering · Chip Huyen, 2025

Chip Huyen. O'Reilly. Used: Ch. 6 "RAG and Agents": "Agent Failure Modes and Evaluation" > "Tool failures".

Site Reliability Engineering · Betsy Beyer et al., 2016

Betsy Beyer, Chris Jones, Jennifer Petoff & Niall Richard Murphy (eds.). O'Reilly. Used: Ch. 22 "Addressing Cascading Failures": "Retries", "Latency and Deadlines" (fetched 2026-10-23). Open source

Timeouts, retries, and backoff with jitter · Marc Brooker (Amazon Builders' Library), 2019

Marc Brooker (Amazon Builders' Library). Used: Timeouts; Retries and backoff; Jitter; Idempotent APIs (fetched 2026-10-23). Open source

Exponential Backoff And Jitter · Marc Brooker (AWS Architecture Blog), 2015

Marc Brooker (AWS Architecture Blog). Used: Conclusions (fetched 2026-10-23). Open source

Idempotent requests (Stripe API reference) · Stripe

Stripe. Used: API v1 behaviour (fetched 2026-10-23). Open source