Junior interview question
How do you choose a model for an application?
Model answer
Filter on hard attributes you can't change (licence, model size, privacy, control), narrow with public benchmarks, run your own evaluation balancing quality, latency and cost, then monitor in production. You want the best model for your application, not the leaderboard.
Scroll sideways to see the whole diagram
AI Engineering: hard attributes are what you can't or won't change, such as licences, training data, privacy policy, or latency of a model someone else hosts. Filter on them first.
Text version
- AI Engineering: hard attributes are what you can't or won't change, such as licences, training data, privacy policy, or latency of a model someone else hosts. Filter on them first.
- Use public benchmarks and leaderboards to narrow the list, while knowing they can't be fully trusted for your task.
- Then run your own evaluation pipeline, balancing quality, latency and cost.
- Keep monitoring in production, and feed failures back into evaluation.
- The book stresses the steps are iterative: you might plan to self-host, then find open models can't reach the quality you need.
Practise the rest
6 Model routing questions, 492 in all, each with a model answer. Free.