Mid-level interview question
When would you use semantic similarity instead of word overlap, and what does it depend on?
Model answer
When many phrasings are right. Embeddings compare meaning, so a rewording can score high without sharing words, and “the dog bit the man” isn't scored as identical to its reverse. But it's only as good as the embedding model, which may not represent your domain, and it costs compute on every output.
Practise the rest
8 Reference scoring questions, 492 in all, each with a model answer. Free.
More Reference scoring interview questions
- What's the difference between exact and subjective evaluation?
- What is pass@k?
- Why can BLEU or ROUGE mislead you when evaluating a model?
- How do you choose a scoring method for a new LLM feature?
- Can perplexity tell you which of two chat models is better?
- Implement AUC-ROC with correct tie handling, a numerically stable softmax, and logistic regression trained with cross-entropy and L2 regularization.