# Quantization

> Quantization: Storing a model's numbers in a format with fewer bits, such as FP16, INT8 or INT4, to cut memory and the bytes each step reads. Strictly it…

Canonical: https://llmdojo.dev/glossary/quantization

Also called quantized, quantize.

Storing a model's numbers in a format with fewer bits, such as FP16, INT8 or INT4, to cut memory and the bytes each step reads. Strictly it means converting to integers, but in practice it covers any precision reduction.

Learn it in the lesson: [Fewer bytes per weight](https://llmdojo.dev/learn/inference-optimization/precision-and-quantization/fewer-bytes-per-weight) (Inference and Model Optimization · Precision and Quantization: Fewer Bytes per Weight).

## Interview questions about Quantization

- [Explain absmax INT8 quantization. What's the worst-case rounding error, and what does a single outlier do?](https://llmdojo.dev/interview/q/explain-absmax-int8-quantization-what-s-the-worst-case-rounding-error-and)
- [You must serve an open 13B model on one 80 GB accelerator and want more concurrent conversations. Which quantization would you use, and what does it actually buy?](https://llmdojo.dev/interview/q/you-must-serve-an-open-13b-model-on-one-80-gb-accelerator)
- [Implement FP32-to-int8 quantization in NumPy with a scale, zero point and clipping, and the matching dequantizer.](https://llmdojo.dev/interview/q/implement-fp32-to-int8-quantization-in-numpy-with-a-scale-zero-point)
- [Explain quantization-aware training, knowledge distillation, eval mode and contrastive learning. How would you verify that each one improved a production model?](https://llmdojo.dev/interview/q/explain-quantization-aware-training-knowledge-distillation-eval-mode-and-contrastive-learning-how)
- [Cover quantization, pruning, distillation and kernel fusion, then compare tensor and pipeline parallelism.](https://llmdojo.dev/interview/q/cover-quantization-pruning-distillation-and-kernel-fusion-then-compare-tensor-and-pipeline)
- [Design and implement a KV cache for autoregressive decoding in a Transformer: tensor shapes for prefill and per-step decode, per-layer caches, batching, memory limits and beam search.](https://llmdojo.dev/interview/q/design-and-implement-a-kv-cache-for-autoregressive-decoding-in-a-transformer)
- [Your AI chatbot takes too long to respond. What do you do?](https://llmdojo.dev/interview/q/your-ai-chatbot-takes-too-long-to-respond-what-do-you-do)
- [Design optimised inference pipelines, and tell me how you would benchmark them.](https://llmdojo.dev/interview/q/design-optimised-inference-pipelines-and-tell-me-how-you-would-benchmark-them)

## Related
- [BF16 (bfloat16)](https://llmdojo.dev/glossary/bf16)
- [Post-training quantization](https://llmdojo.dev/glossary/ptq)
- [Quantization-aware training](https://llmdojo.dev/glossary/qat)
- [Glossary of every term](https://llmdojo.dev/glossary)