# Layer norm

> Layer norm: Rescales each token's vector to a standard spread before a sub-layer, which keeps training stable.

Canonical: https://llmdojo.dev/glossary/layer-norm

Also called layer normalization.

Rescales each token's vector to a standard spread before a sub-layer, which keeps training stable.

Learn it in the lesson: [The transformer block](https://llmdojo.dev/learn/model-foundations/attention/the-transformer-block) (Model Foundations · Attention: How Tokens Use Their Context).

## Interview questions about Layer norm

- [Compare batch normalization and layer normalization: which axes does each normalize over, how do they differ between training and inference, and how would you debug a normalization bug?](https://llmdojo.dev/interview/q/compare-batch-normalization-and-layer-normalization-which-axes-does-each-normalize-over)
- [What's inside one transformer block, and where do most of its parameters sit?](https://llmdojo.dev/interview/q/what-s-inside-one-transformer-block-and-where-do-most-of-its)

## Related
- [Attention](https://llmdojo.dev/glossary/attention)
- [Query, key and value](https://llmdojo.dev/glossary/qkv)
- [Attention head](https://llmdojo.dev/glossary/attention-head)
- [Causal mask](https://llmdojo.dev/glossary/causal-mask)
- [Softmax](https://llmdojo.dev/glossary/softmax)
- [Transformer block](https://llmdojo.dev/glossary/transformer-block)
- [Residual connection](https://llmdojo.dev/glossary/residual-connection)
- [Feed-forward network (MLP)](https://llmdojo.dev/glossary/mlp)
- [Glossary of every term](https://llmdojo.dev/glossary)