Topic 6

Every failure becomes a test

A fixed failure should stay fixed. The way to make sure is to turn each failure you find into a test case, part of the set of examples the system is known to get wrong. AI Engineering (Ch. 4, "Annotate evaluation data") recommends keeping exactly that set alongside the production-like one.

Predict first

You fix the out-of-date price. What stops the same failure from coming back next month?

Nothing: you'd noticeA note in the docsA test case for it, with the expected answer and a check, run on every changeA bigger model

From a failure to a test caseStep 1 / 5

Scroll sideways to see the whole diagram

Error analysis turns up a real failure.

Text version

Error analysis turns up a real failure.
Write it down as a test case: the same input, the right answer, and a check that can run without you.
It joins the set of cases the system is known to get wrong, one of lesson 1's evaluation sets.
That set runs on every change, as in Module 03's regression suite.
Now a fix is checked forever, and a change that brings the failure back is caught.

Module 03's regression suite is where these cases run. Each round of error analysis adds a few, so the test set grows towards the failures your system actually has, not the ones you imagined at the start. It also keeps the size question from lesson 4 honest: as the set grows, smaller differences become measurable.

Two cautions. Keep the original inputs, not cleaned-up versions, because the messiness is often why they failed. And don't only add failures: if every case is a known failure, the overall pass rate no longer reflects typical traffic. Keep the production-like set separate.

Quick check

Why keep the known-failures set separate from the production-like set?

To make the numbers look betterBecause they use different scorersSo each answers its own question: whether old failures stay fixed, and how the system does on typical trafficIt doesn't matter