# BF16 (bfloat16)

> BF16 (bfloat16): A 16-bit float with 1 sign, 8 exponent and 7 fraction bits. It has the same range as FP32 but less precision than FP16.

Canonical: https://llmdojo.dev/glossary/bf16

Also called bf16, bfloat16.

A 16-bit float with 1 sign, 8 exponent and 7 fraction bits. It has the same range as FP32 but less precision than FP16.

Learn it in the lesson: [Range versus precision](https://llmdojo.dev/learn/inference-optimization/precision-and-quantization/range-and-precision) (Inference and Model Optimization · Precision and Quantization: Fewer Bytes per Weight).

## Interview questions about BF16 (bfloat16)

- [FP16 and BF16 both use 16 bits. How do they differ, and how can loading a model in the wrong one hurt?](https://llmdojo.dev/interview/q/fp16-and-bf16-both-use-16-bits-how-do-they-differ-and)
- [Walk me through Transformer attention: causal and padding masks, cross-attention, why a KV cache helps at inference, and how mixed-precision training works.](https://llmdojo.dev/interview/q/walk-me-through-transformer-attention-causal-and-padding-masks-cross-attention-why)
- [How would you optimise large-scale training and inference on GPUs?](https://llmdojo.dev/interview/q/how-would-you-optimise-large-scale-training-and-inference-on-gpus)
- [Estimate an FFN layer's FLOPs, parameter and activation bytes, and the communication from sharding it. Which limit binds: compute, HBM or network?](https://llmdojo.dev/interview/q/estimate-an-ffn-layer-s-flops-parameter-and-activation-bytes-and-the)
- [Estimate the GPU memory a very large matmul needs, then compare pipeline and tensor parallelism for it.](https://llmdojo.dev/interview/q/estimate-the-gpu-memory-a-very-large-matmul-needs-then-compare-pipeline)

## Related
- [Quantization](https://llmdojo.dev/glossary/quantization)
- [Post-training quantization](https://llmdojo.dev/glossary/ptq)
- [Quantization-aware training](https://llmdojo.dev/glossary/qat)
- [Glossary of every term](https://llmdojo.dev/glossary)