Senior interview questionAsked at Scale AI · interview write-up
You're handed fine-tuning code whose loss behaves oddly. What do you check?
Model answer
Start with what the model actually sees: decode a batch back to text. Usual suspects: a chat template that differs from the base model's; labels shifted twice or not at all; a loss not masked to the answer tokens, so the model learns to predict prompts; padding counted in the loss; new special tokens missing from the tokenizer; a learning rate too high for the batch. Then overfit ten examples: if the loss won't approach zero, the bug is in the pipeline, not the data.
Practise the rest
20 Fine-tuning & alignment questions, 492 in all, each with a model answer. Free.
More from Scale AI interview questions.
More Fine-tuning & alignment interview questions
- Prompting, RAG or fine-tuning: how do you choose?
- What is LoRA, and why is it so much cheaper than full fine-tuning?
- Compare RLHF with a reward model to DPO.
- Debug a GRPO training loop. What is the probability ratio, and what goes wrong with it?
- Design a platform for fine-tuning LLMs on AWS, covering distributed training and security between tenants.
- How do you detect and prevent reward hacking during LLM post-training?