# Decode

> Decode: The sequential phase of generation: one forward pass per new token.

Canonical: https://llmdojo.dev/glossary/decode

Also called decode steps, decode step, decode phase.

The sequential phase of generation: one forward pass per new token.

Learn it in the lesson: [Prefill and decode](https://llmdojo.dev/learn/model-foundations/context/prefill-and-decode) (Model Foundations · Context Windows, Prefill and the KV Cache).

## Interview questions about Decode

- [Implement a byte-pair encoding tokenizer with train, encode and decode, where training keeps merging the most frequent adjacent pair until counts fall below a threshold.](https://llmdojo.dev/interview/q/implement-a-byte-pair-encoding-tokenizer-with-train-encode-and-decode-where)
- [Users say your summariser 'takes ages to start' on long documents but then writes quickly. Explain what's happening in terms of prefill and decode.](https://llmdojo.dev/interview/q/users-say-your-summariser-takes-ages-to-start-on-long-documents-but)
- [Design and implement a KV cache for autoregressive decoding in a Transformer: tensor shapes for prefill and per-step decode, per-layer caches, batching, memory limits and beam search.](https://llmdojo.dev/interview/q/design-and-implement-a-kv-cache-for-autoregressive-decoding-in-a-transformer)
- [What's the difference between compute-bound and memory bandwidth-bound work, and which one is decode?](https://llmdojo.dev/interview/q/what-s-the-difference-between-compute-bound-and-memory-bandwidth-bound-work)
- [How would you show, on a given machine, that decode is bandwidth-bound and prefill compute-bound?](https://llmdojo.dev/interview/q/how-would-you-show-on-a-given-machine-that-decode-is-bandwidth)
- [Why does decoding a batch of 8 cost far less than 8 separate decode steps, and why can't the batch grow forever?](https://llmdojo.dev/interview/q/why-does-decoding-a-batch-of-8-cost-far-less-than-8)
- [A colleague says INT4 weights will make decode eight times faster because they're eight times smaller. How do you respond?](https://llmdojo.dev/interview/q/a-colleague-says-int4-weights-will-make-decode-eight-times-faster-because)
- [Walk me through Transformer attention: causal and padding masks, cross-attention, why a KV cache helps at inference, and how mixed-precision training works.](https://llmdojo.dev/interview/q/walk-me-through-transformer-attention-causal-and-padding-masks-cross-attention-why)

## Related
- [Context window](https://llmdojo.dev/glossary/context-window)
- [Position embedding](https://llmdojo.dev/glossary/position-embedding)
- [Prefill](https://llmdojo.dev/glossary/prefill)
- [KV cache](https://llmdojo.dev/glossary/kv-cache)
- [Glossary of every term](https://llmdojo.dev/glossary)