Senior interview questionAsked at Cohere · interview write-up
An enterprise customer wants a 7B model that's better at coding in their codebase. Design the post-training pipeline.
Model answer
Start with the target: an execution-based eval built from the customer's real tasks, with unit tests that pass or fail, plus public coding benchmarks to catch regressions.
Data: their repositories, internal libraries and review history, scrubbed of secrets, plus synthetic tasks kept only if they run. Then stages: continued pretraining on their code if it's unusual, SFT on instruction and solution pairs, then RL from execution feedback or DPO on passing versus failing samples. Gate every stage on the eval and watch for forgetting. LoRA adapters keep many customers on one base.
Practise the rest
20 Fine-tuning & alignment questions, 492 in all, each with a model answer. Free.
More from Cohere interview questions.
More Fine-tuning & alignment interview questions
- Prompting, RAG or fine-tuning: how do you choose?
- What is LoRA, and why is it so much cheaper than full fine-tuning?
- You're handed fine-tuning code whose loss behaves oddly. What do you check?
- Compare RLHF with a reward model to DPO.
- Debug a GRPO training loop. What is the probability ratio, and what goes wrong with it?
- Design a platform for fine-tuning LLMs on AWS, covering distributed training and security between tenants.