Your answers must start within one second. How do you budget it?

Add up the stages before the first token: router, retrieval, reranker, the model's time to first token. Streaming hides generation, but not for answers needed whole, like JSON or output-guardrail checks.

Parallelise independent stages, skip optional ones under load, and budget percentiles, not averages.