# Test set

> Test set: A fixed collection of inputs with known expected answers, run unchanged against every version so their results can be compared case by case.

Canonical: https://llmdojo.dev/glossary/test-set

Also called test sets, evaluation set, evaluation sets.

A fixed collection of inputs with known expected answers, run unchanged against every version so their results can be compared case by case.

Learn it in the lesson: [A fixed test set](https://llmdojo.dev/learn/prompt-context-engineering/prompt-versioning/a-fixed-test-set) (Prompt and Context Engineering · Versioning Prompts and Catching Regressions).

## Interview questions about Test set

- [For a medical conversational AI, how would you source and split data, choose between prompting, fine-tuning and RAG, and evaluate for hallucination and unsafe advice?](https://llmdojo.dev/interview/q/for-a-medical-conversational-ai-how-would-you-source-and-split-data)
- [Your few-shot sentiment prompt lists six positive examples, then two negative ones at the very end. What's the risk, and what would you do?](https://llmdojo.dev/interview/q/your-few-shot-sentiment-prompt-lists-six-positive-examples-then-two-negative)
- [Version B has a higher pass rate than version A on last week's production logs. Do you ship B?](https://llmdojo.dev/interview/q/version-b-has-a-higher-pass-rate-than-version-a-on-last)
- [What is benchmark contamination, and how would you detect it?](https://llmdojo.dev/interview/q/what-is-benchmark-contamination-and-how-would-you-detect-it)
- [A reranker fixes most of your misses but still prefers “safe for children over three” to “not safe for children under three”. What do you conclude?](https://llmdojo.dev/interview/q/a-reranker-fixes-most-of-your-misses-but-still-prefers-safe-for)
- [Anthropic reports large gains from contextual retrieval. What exactly did they measure, and what does it cost?](https://llmdojo.dev/interview/q/anthropic-reports-large-gains-from-contextual-retrieval-what-exactly-did-they-measure)
- [How do you set a difficulty router's threshold?](https://llmdojo.dev/interview/q/how-do-you-set-a-difficulty-router-s-threshold)
- [Metrics are stable, yet you suspect drift. What drifts in an AI system?](https://llmdojo.dev/interview/q/metrics-are-stable-yet-you-suspect-drift-what-drifts-in-an-ai)

## Related
- [Prompt version](https://llmdojo.dev/glossary/prompt-version)
- [Exact match](https://llmdojo.dev/glossary/exact-match)
- [Glossary of every term](https://llmdojo.dev/glossary)