Mid-level interview questionAsked at NVIDIA · interview write-up
Implement FP32-to-int8 quantization in NumPy with a scale, zero point and clipping, and the matching dequantizer.
Model answer
import numpy as np
def quantize(x, qmin=-128, qmax=127):
lo, hi = min(x.min(), 0.0), max(x.max(), 0.0)
scale = (hi - lo) / (qmax - qmin) or 1.0
zp = int(np.clip(round(qmin - lo / scale),
qmin, qmax))
q = np.round(x / scale) + zp
return np.clip(q, qmin, qmax).astype(np.int8), scale, zp
def dequantize(q, scale, zp):
return (q.astype(np.float32) - zp) * scale
Asymmetric quantization maps the observed range onto -128 to 127 with a scale and a zero point, keeping 0.0 exactly representable. Interviewers check rounding before adding the zero point, clipping, the all-zero case, and outliers: one huge value stretches the scale and crushes the rest, so clip to a percentile or use per-channel scales.
Practise the rest
7 Quantization questions, 492 in all, each with a model answer. Free.
More from NVIDIA interview questions.
More Quantization interview questions
- How much memory do a model's weights take, and what does quantizing GPT-2 small to INT4 save?
- FP16 and BF16 both use 16 bits. How do they differ, and how can loading a model in the wrong one hurt?
- Explain absmax INT8 quantization. What's the worst-case rounding error, and what does a single outlier do?
- You must serve an open 13B model on one 80 GB accelerator and want more concurrent conversations. Which quantization would you use, and what does it actually buy?
- A colleague says INT4 weights will make decode eight times faster because they're eight times smaller. How do you respond?
- Explain quantization-aware training, knowledge distillation, eval mode and contrastive learning. How would you verify that each one improved a production model?