Junior interview questionAsked at Waymo · interview write-up
You only have small counts. How would you statistically compare two rare-event detection models and decide which is better?
Model answer
Accuracy hides rare events; compare counts on the same cases. If both models saw the same data, use McNemar's paired test, which uses only cases where they disagree. For rates like detections per thousand miles, compare Poisson rates with an exact test. With few events, report the wide intervals too.
Practise the rest
7 Comparing systems questions, 492 in all, each with a model answer. Free.
More from Waymo interview questions.
More Comparing systems interview questions
- Why does every evaluation result need a baseline?
- What is an ablation study, and when would you run one?
- Version B scores 4 points higher than A on 100 test cases. Is B better?
- What is benchmark contamination, and how would you detect it?
- How many test cases do you need to compare two LLM systems?
- How should you use public leaderboards like Chatbot Arena when choosing a model?