Topic 6
Capstone: a production-readiness plan
This module's capstone is a production-readiness plan: for an AI feature you're building or running, what's in place, and what isn't yet.
Predict first
Is it ready for production?
Capstone · tick what your system already has; the rest is your plan
Your plan: 11 open items
- Models chosen by our own evaluation, versions pinned, routing measured (lesson)
- A gateway controls access, keys and usage (lesson)
- A fallback model that fails independently, behind a circuit breaker (lesson)
- Degraded modes and user-facing messages, tested on a schedule (lesson)
- Cost, rate and latency budgets, enforced per user and per request (lesson)
- Spend tracked daily, with alerts before the cap (lesson)
- Metrics designed from failure modes, broken down by release and segment (lesson)
- Traces for every request; an incident and postmortem process (lesson)
- Input and output guardrails, with false refusals tracked (lesson)
- Changes ship behind flags, through shadow and canary (lesson)
- Failures classified; fine-tuning only for behaviour that prompting and RAG can't fix (lesson)
Not a score: a list of gaps, each linked to the lesson that covers it. A real plan adds an owner and a date to every open line.
Each line maps to a lesson in this module. Tick what your system genuinely has, with evidence, not memory: a dashboard you can open, a flag you've actually flipped, a fallback you've seen serve traffic. What's left is your plan.
To turn it into a document, add for each open line:
- the risk of leaving it open: what failure would it let through, and how often;
- the smallest version that closes most of the risk (often a day's work, not a project);
- an owner and a date;
- the evidence that will show it's done: a test, a metric, a drill.
Then rank the lines the way Module 04's error analysis ranked failures: by how likely and how harmful each gap is, against what it costs to close.
Quick check