Guardrails and safe deployment interview questions
6 questions with model answers
- junior 1
- mid 3
- senior 2
6 guardrails and safe deployment questions asked in AI engineer, ML engineer and applied AI engineer interviews, each with a model answer you can say out loud in a minute or two: 1 junior, 3 mid-level and 2 senior. The lesson Guardrails and Safe Deployment teaches the ideas behind them.Updated
Model answer
Input guardrails stop private data leaking to external APIs and bad prompts compromising the system. Output guardrails catch failures (format, facts, toxicity, private data, brand risk) and apply a policy: retry, repair, block or hand to a person.
Scroll sideways to see the whole diagram
AI Engineering: input guardrails protect against two risks: leaking private information to external APIs, and executing prompts that compromise your system.
Text version
- AI Engineering: input guardrails protect against two risks: leaking private information to external APIs, and executing prompts that compromise your system.
- The model runs on whatever passes.
- Output guardrails have two jobs: catch output failures, and specify the policy for handling each kind.
Model answer
Detect it in the assembled prompt, including retrieved passages and tool results, then block or mask it with placeholders, keeping a reverse map to restore values in the reply. It isn't airtight: detectors miss things, and the model can't use what it can't see.
Model answer
Count both errors on a labelled sample: harmful outputs let through and good outputs blocked. The book stresses tracking false refusals: a system can be too secure. Pick the threshold whose mistakes you can best afford.
Model answer
Partial responses are hard to evaluate, so an unsafe part may already be on screen; holding the answer for a check gives up streaming's fast first token. Choose per feature: stream casual chat, hold back high-stakes answers.
Scroll sideways to see the whole diagram
AI Engineering: in streaming mode, partial responses are hard to evaluate, so unsafe content may reach users before a guardrail can block it.
Text version
- AI Engineering: in streaming mode, partial responses are hard to evaluate, so unsafe content may reach users before a guardrail can block it.
- Checking the full response first is safer but slower. The book reports some teams skip guardrails because of the latency.
- The choice can differ by feature: stream a casual chat, hold back a medical summary until it's checked.
- Providers ship guardrails of their own, which reduces what you must build on third-party APIs; self-hosting removes some input risks (nothing leaves) but means building more yourself.
Model answer
Offline evaluation and regression suite; behind a feature flag so turning it off is seconds, not a deploy; shadow on copies of traffic; a small canary compared with control on a few user-facing metrics; then ramp in steps.
Model answer
Exposure scales with share: the workbook's example is a 5% canary at 20% errors adding about 1% overall. Comparing canary with control on the same days also catches problems sooner. In the course's illustrative example, 25 extra bad answers instead of 1,500.