Senior interview questionAsked at Cohere · interview write-up
A model struggles with multi-step reasoning on a specific domain task. How would you improve it?
Model answer
First measure: a domain eval with checkable answers, and read the failures to tell missing knowledge from bad reasoning. Missing facts call for retrieval, not training.
Cheap wins next: chain-of-thought with worked domain examples, and self-consistency, sampling several chains and voting. If that isn't enough, train: SFT on reasoning traces from experts or a stronger model, kept only when the answer is correct, then RL with verifiable rewards, as in GRPO. Watch for reward hacking and chains that grow without getting better.
Practise the rest
20 Fine-tuning & alignment questions, 492 in all, each with a model answer. Free.
More from Cohere interview questions.
More Fine-tuning & alignment interview questions
- Prompting, RAG or fine-tuning: how do you choose?
- What is LoRA, and why is it so much cheaper than full fine-tuning?
- You're handed fine-tuning code whose loss behaves oddly. What do you check?
- Compare RLHF with a reward model to DPO.
- Debug a GRPO training loop. What is the probability ratio, and what goes wrong with it?
- Design a platform for fine-tuning LLMs on AWS, covering distributed training and security between tenants.