When to fine-tune interview questions
6 questions with model answers
- junior 1
- mid 3
- senior 2
6 when to fine-tune questions asked in AI engineer, ML engineer and applied AI engineer interviews, each with a model answer you can say out loud in a minute or two: 1 junior, 3 mid-level and 2 senior. The lesson When to Fine-Tune, and a Production-Readiness Plan teaches the ideas behind them.Updated
Model answer
Usually after systematic prompting, examples and retrieval. Fine-tuning needs far more data, compute and expertise. It's for behaviour failures prompting can't fix. Set up evaluation first so you know whether it helped.
Scroll sideways to see the whole diagram
AI Engineering: before any adaptation, define your evaluation criteria and pipeline. Every later step is measured with it.
Text version
- AI Engineering: before any adaptation, define your evaluation criteria and pipeline. Every later step is measured with it.
- Start with prompting alone, then add examples to the prompt: the book suggests anywhere from 1 to 50.
- If the model fails for lack of information, connect it to data, starting with basic retrieval such as term-based search (Module 05).
- Then follow the failures: information failures call for better retrieval; behaviour failures (irrelevant, malformatted or unsafe outputs) for fine-tuning.
- The two can be combined. The book notes better retrieval adds inference complexity, while fine-tuning adds development complexity but leaves inference unchanged.
Model answer
Classify the failures. Information failures (missing or outdated facts) call for RAG; behaviour failures (wrong format, style or relevance) for fine-tuning. The book: fine-tuning is for form, RAG is for facts. With both, start with RAG.
Model answer
Output structure or a rare syntax, like an uncommon SQL dialect; task-specific behaviour; bias mitigation; and a small, specialised model that's cheaper to serve, possibly distilled. The book's example: Grammarly's fine-tuned Flan-T5 beat a GPT-3 text-editing variant at 60× smaller.
Model answer
Besides the weights, you store gradients and optimizer states for every trained parameter: a 7B model in 16-bit with Adam is 14 GB plus 42 GB. LoRA trains small low-rank matrices instead, merged back without inference latency, so training memory shrinks.
Scroll sideways to see the whole diagram
The pretrained weight matrix W stays frozen: no gradients, no optimizer state for it.
Text version
- The pretrained weight matrix W stays frozen: no gradients, no optimizer state for it.
- LoRA adds a thin side path. A squeezes the input down to r numbers and B expands it back; only A and B are trained.
- After training, B·A can be added into W, so the fine-tuned model runs exactly as fast as the original.
Hu et al. (2021), LoRA: Low-Rank Adaptation of Large Language Models. Counts are for one 4,096 × 4,096 matrix at rank 8.
Model answer
It can degrade other tasks; it needs data, expertise, serving and maintenance as new base models arrive. Often prompt experiments weren't systematic. The book's cautionary tale: BloombergGPT, about 1.3M A100 hours, was outperformed on financial benchmarks by GPT-4-0314, released the same month.
Model answer
Check each layer with evidence: evaluated and pinned models, routing and a gateway; independent fallbacks and degraded modes; enforced budgets; metrics, traces and an incident process; guardrails and flagged, canaried releases. Turn gaps into a list ranked by likelihood × harm against effort, each with an owner, a date and the evidence that will show it's done.