Mid-level interview question
Why don't output guardrails mix well with streaming?
Model answer
Partial responses are hard to evaluate, so an unsafe part may already be on screen; holding the answer for a check gives up streaming's fast first token. Choose per feature: stream casual chat, hold back high-stakes answers.
Scroll sideways to see the whole diagram
AI Engineering: in streaming mode, partial responses are hard to evaluate, so unsafe content may reach users before a guardrail can block it.
Text version
- AI Engineering: in streaming mode, partial responses are hard to evaluate, so unsafe content may reach users before a guardrail can block it.
- Checking the full response first is safer but slower. The book reports some teams skip guardrails because of the latency.
- The choice can differ by feature: stream a casual chat, hold back a medical summary until it's checked.
- Providers ship guardrails of their own, which reduces what you must build on third-party APIs; self-hosting removes some input risks (nothing leaves) but means building more yourself.
Practise the rest
6 Guardrails questions, 492 in all, each with a model answer. Free.