Mid-level interview questionAsked by candidates · YouTube
What is LoRA, and why is it so much cheaper than full fine-tuning?
Model answer
LoRA freezes the original weights and learns a small low-rank update for chosen matrices: W + BA, where B and A are thin, rank 8 or 16, say.
Scroll sideways to see the whole diagram
The pretrained weight matrix W stays frozen: no gradients, no optimizer state for it.
Text version
- The pretrained weight matrix W stays frozen: no gradients, no optimizer state for it.
- LoRA adds a thin side path. A squeezes the input down to r numbers and B expands it back; only A and B are trained.
- After training, B·A can be added into W, so the fine-tuned model runs exactly as fast as the original.
Hu et al. (2021), LoRA: Low-Rank Adaptation of Large Language Models. Counts are for one 4,096 × 4,096 matrix at rank 8.
Only A and B train, so gradients and optimizer state are a tiny fraction of the model's, and one base model can serve many adapters. After training you can merge BA into W, so inference costs nothing extra.
Practise the rest
20 Fine-tuning & alignment questions, 492 in all, each with a model answer. Free.
More Fine-tuning & alignment interview questions
- Prompting, RAG or fine-tuning: how do you choose?
- You're handed fine-tuning code whose loss behaves oddly. What do you check?
- Compare RLHF with a reward model to DPO.
- Debug a GRPO training loop. What is the probability ratio, and what goes wrong with it?
- Design a platform for fine-tuning LLMs on AWS, covering distributed training and security between tenants.
- How do you detect and prevent reward hacking during LLM post-training?