Post-training quantization
Also called ptq
Quantizing a model after it has been fully trained. It is the most common approach, and the one application developers use.
Interview questions about Post-training quantization
- You must serve an open 13B model on one 80 GB accelerator and want more concurrent conversations. Which quantization would you use, and what does it actually buy?
Post-training, weight-only quantization. I didn't train the model, PTQ is the common path frameworks offer in a few lines, and weight quantization affects…
- Explain quantization-aware training, knowledge distillation, eval mode and contrastive learning. How would you verify that each one improved a production model?
Quantization-aware training simulates low-precision rounding during fine-tuning, so accuracy holds better than post-training quantization.