Junior interview question
Why does every evaluation result need a baseline?
Model answer
A score means little until you know what it beats. Random guessing is the floor (25% on four-option multiple choice), then the simplest approach that could work, the version you run today, and sometimes a person doing the job. Most real decisions are against what you'd replace.
Practise the rest
7 Comparing systems questions, 492 in all, each with a model answer. Free.
More Comparing systems interview questions
- What is an ablation study, and when would you run one?
- Version B scores 4 points higher than A on 100 test cases. Is B better?
- What is benchmark contamination, and how would you detect it?
- How many test cases do you need to compare two LLM systems?
- How should you use public leaderboards like Chatbot Arena when choosing a model?
- You only have small counts. How would you statistically compare two rare-event detection models and decide which is better?