Mid-level interview questionAsked at Waymo · interview write-up
Explain quantization-aware training, knowledge distillation, eval mode and contrastive learning. How would you verify that each one improved a production model?
Model answer
Quantization-aware training simulates low-precision rounding during fine-tuning, so accuracy holds better than post-training quantization. Distillation trains a small student on a teacher's soft outputs. Eval mode disables dropout and makes batch norm use running statistics; forgetting it corrupts metrics. Contrastive learning pulls matching pairs together in embedding space and pushes others apart. Verify each on a fixed held-out set and the target hardware: accuracy per slice, latency and memory versus the baseline.
Practise the rest
7 Quantization questions, 492 in all, each with a model answer. Free.
More from Waymo interview questions.
More Quantization interview questions
- How much memory do a model's weights take, and what does quantizing GPT-2 small to INT4 save?
- FP16 and BF16 both use 16 bits. How do they differ, and how can loading a model in the wrong one hurt?
- Explain absmax INT8 quantization. What's the worst-case rounding error, and what does a single outlier do?
- You must serve an open 13B model on one 80 GB accelerator and want more concurrent conversations. Which quantization would you use, and what does it actually buy?
- A colleague says INT4 weights will make decode eight times faster because they're eight times smaller. How do you respond?
- Implement FP32-to-int8 quantization in NumPy with a scale, zero point and clipping, and the matching dequantizer.