# Transformer block

> Transformer block: The repeated unit of a transformer: attention and a feed-forward network, each wrapped in a layer norm and a residual connection.

Canonical: https://llmdojo.dev/glossary/transformer-block

Also called transformer blocks.

The repeated unit of a transformer: attention and a feed-forward network, each wrapped in a layer norm and a residual connection.

Learn it in the lesson: [The transformer block](https://llmdojo.dev/learn/model-foundations/attention/the-transformer-block) (Model Foundations · Attention: How Tokens Use Their Context).

## Interview questions about Transformer block

- [What's inside one transformer block, and where do most of its parameters sit?](https://llmdojo.dev/interview/q/what-s-inside-one-transformer-block-and-where-do-most-of-its)
- [A pre-norm transformer block trains badly. What do you check around the norms and residuals?](https://llmdojo.dev/interview/q/a-pre-norm-transformer-block-trains-badly-what-do-you-check-around)
- [Implement scaled dot-product multi-head attention and a Transformer block from scratch, including the backward pass for attention, without autograd.](https://llmdojo.dev/interview/q/implement-scaled-dot-product-multi-head-attention-and-a-transformer-block-from)

## Related
- [Attention](https://llmdojo.dev/glossary/attention)
- [Query, key and value](https://llmdojo.dev/glossary/qkv)
- [Attention head](https://llmdojo.dev/glossary/attention-head)
- [Causal mask](https://llmdojo.dev/glossary/causal-mask)
- [Softmax](https://llmdojo.dev/glossary/softmax)
- [Residual connection](https://llmdojo.dev/glossary/residual-connection)
- [Layer norm](https://llmdojo.dev/glossary/layer-norm)
- [Feed-forward network (MLP)](https://llmdojo.dev/glossary/mlp)
- [Glossary of every term](https://llmdojo.dev/glossary)