Junior interview question
When should you fine-tune instead of prompting?
Model answer
Usually after systematic prompting, examples and retrieval. Fine-tuning needs far more data, compute and expertise. It's for behaviour failures prompting can't fix. Set up evaluation first so you know whether it helped.
Scroll sideways to see the whole diagram
AI Engineering: before any adaptation, define your evaluation criteria and pipeline. Every later step is measured with it.
Text version
- AI Engineering: before any adaptation, define your evaluation criteria and pipeline. Every later step is measured with it.
- Start with prompting alone, then add examples to the prompt: the book suggests anywhere from 1 to 50.
- If the model fails for lack of information, connect it to data, starting with basic retrieval such as term-based search (Module 05).
- Then follow the failures: information failures call for better retrieval; behaviour failures (irrelevant, malformatted or unsafe outputs) for fine-tuning.
- The two can be combined. The book notes better retrieval adds inference complexity, while fine-tuning adds development complexity but leaves inference unchanged.
Practise the rest
6 When to fine-tune questions, 492 in all, each with a model answer. Free.