Topic 2
Good reasons to fine-tune
AI Engineering (Ch. 7, "Reasons to Finetune") gives the primary reason simply: to improve a model's quality, in general or on a specific task. Its examples:
- Output structure. Fine-tuning is commonly used to get outputs that follow specific formats, such as JSON or YAML.
- Tasks the model wasn't trained enough on. A model may handle standard SQL but fail on a less common dialect, or on customer-specific queries.
- Bias mitigation. Curated data can counteract biases a model picked up in pre-training.
- Small, specialised models. Fine-tuning small models is far more common than fine-tuning large ones: they need less memory and are cheaper and faster to serve. Distillation trains a small model to imitate a larger one using the larger model's outputs.
Predict first
Scroll sideways to see the whole diagram
AI Engineering: fine-tuning is commonly used to make outputs follow specific structures, and helps where a model wasn't trained enough on your task, such as a less common SQL dialect.
Text version
- AI Engineering: fine-tuning is commonly used to make outputs follow specific structures, and helps where a model wasn't trained enough on your task, such as a less common SQL dialect.
- Behaviour issues: outputs that are correct but irrelevant, like technical specs missing the detail your engineers need.
- Fine-tuning smaller models is more common than fine-tuning big ones: cheaper and faster in production. Distillation trains a small model on a larger model's outputs. The book also describes fine-tuning to counter biases.
- The book's example of a small specialised model beating a much larger general one.
Grammarly figures as reported in AI Engineering (Huyen), Ch. 7 "Reasons to Finetune".
There's also a cost argument, with a caveat. Before prompt caching existed, the book notes, one benefit of fine-tuning was token savings: instead of putting many examples in every prompt, you could fine-tune on them and use shorter prompts. Prompt caching now covers much of that saving for a long, repeated prefix, without training anything.
Quick check