# Feed-forward network (MLP)

> Feed-forward network (MLP): The part of a transformer block that processes each token's vector on its own, after attention has mixed information between…

Canonical: https://llmdojo.dev/glossary/mlp

Also called feed-forward network, MLP.

The part of a transformer block that processes each token's vector on its own, after attention has mixed information between tokens.

Learn it in the lesson: [The transformer block](https://llmdojo.dev/learn/model-foundations/attention/the-transformer-block) (Model Foundations · Attention: How Tokens Use Their Context).

## Interview questions about Feed-forward network (MLP)

- [Design a streaming system that tokenizes arriving text, embeds it, trains a classifier and serves low-latency predictions.](https://llmdojo.dev/interview/q/design-a-streaming-system-that-tokenizes-arriving-text-embeds-it-trains-a)
- [What's inside one transformer block, and where do most of its parameters sit?](https://llmdojo.dev/interview/q/what-s-inside-one-transformer-block-and-where-do-most-of-its)
- [A pre-norm transformer block trains badly. What do you check around the norms and residuals?](https://llmdojo.dev/interview/q/a-pre-norm-transformer-block-trains-badly-what-do-you-check-around)
- [With no starter code, implement a GPT-style transformer decoder from scratch in about 30 minutes.](https://llmdojo.dev/interview/q/with-no-starter-code-implement-a-gpt-style-transformer-decoder-from-scratch)
- [Implement scaled dot-product multi-head attention and a Transformer block from scratch, including the backward pass for attention, without autograd.](https://llmdojo.dev/interview/q/implement-scaled-dot-product-multi-head-attention-and-a-transformer-block-from)
- [Explain transformer and mixture-of-experts fundamentals, including how training differs from inference.](https://llmdojo.dev/interview/q/explain-transformer-and-mixture-of-experts-fundamentals-including-how-training-differs-from)
- [How does a vision encoder feed images into a multimodal LLM, and where are the bottlenecks?](https://llmdojo.dev/interview/q/how-does-a-vision-encoder-feed-images-into-a-multimodal-llm-and)

## Related
- [Attention](https://llmdojo.dev/glossary/attention)
- [Query, key and value](https://llmdojo.dev/glossary/qkv)
- [Attention head](https://llmdojo.dev/glossary/attention-head)
- [Causal mask](https://llmdojo.dev/glossary/causal-mask)
- [Softmax](https://llmdojo.dev/glossary/softmax)
- [Transformer block](https://llmdojo.dev/glossary/transformer-block)
- [Residual connection](https://llmdojo.dev/glossary/residual-connection)
- [Layer norm](https://llmdojo.dev/glossary/layer-norm)
- [Glossary of every term](https://llmdojo.dev/glossary)