Junior interview question
What kinds of criteria would you use to evaluate an LLM application?
Model answer
Four kinds, scored separately: domain capability (does it understand the subject?), generation (coherent, faithful, safe), instruction following (the requested format and length), and cost and latency. A summary can be faithful but too long, so one overall score would hide what to fix.
Practise the rest
10 Success criteria questions, 492 in all, each with a model answer. Free.
More Success criteria interview questions
- What is evaluation-driven development?
- Two annotators grading the same chatbot replies keep disagreeing. What do you change?
- Your support bot's factual consistency is 90%. Is that good enough to automate support?
- Version B has a higher pass rate than version A on last week's production logs. Do you ship B?
- How would you design the evaluation for a two-step résumé parser: PDF to text, then text to current employer?
- The prompt mixes a regression target with precision and recall: untangle it, and pick metrics and a modeling plan tied to the real decision.