Mid-level interview questionAsked at Zoox · interview write-up
Design simulation experiments to evaluate a self-driving agent's safety. Which scene parameters do you vary, how do you sample them, and how do you estimate a rare collision rate without bias?
Model answer
Vary what drives risk: actor types and behaviours, speeds and gaps, occlusions, road geometry, weather and sensor noise. Sample from real-world distributions fitted to driving logs, so results mean something. Collisions are rare, so plain Monte Carlo wastes runs; use importance sampling to oversample dangerous regions, then weight each run by its likelihood ratio, keeping the collision-rate estimate unbiased, with confidence intervals. Validate the simulator against logged events.
Practise the rest
11 Benchmarking questions, 492 in all, each with a model answer. Free.
More from Zoox interview questions.
More Benchmarking interview questions
- What does a fair inference benchmark hold fixed?
- An upgrade makes your 960-token prefill 5% faster in one before-and-after run. Do you ship it?
- A vendor says their GPU is twice as fast as your CPU. How would you judge what it buys your workload?
- A provider's status page shows a p50 TTFT of 180 ms. You need 300 ms for 9 in 10 of your users. What do you do before signing up?
- In the capstone, continuous batching with at most 8 seats gives a p90 first token of 122 ms against a 300 ms target, yet the planner marks it failing. Why, how do you fix it, and what would you measure first on the real system?
- What does perplexity measure, how do tokenizers distort it, and where does it break down as a quality metric?