# Autoregressive generation

> Autoregressive generation: Writing one token at a time, appending each to the input before predicting the next.

Canonical: https://llmdojo.dev/glossary/autoregressive

Also called autoregressive.

Writing one token at a time, appending each to the input before predicting the next.

Learn it in the lesson: [Generation is a loop](https://llmdojo.dev/learn/model-foundations/sampling/the-generation-loop) (Model Foundations · Next-Token Prediction and Sampling).

## Interview questions about Autoregressive generation

- [Cover the fundamentals: self- versus cross-attention, autoregressive decoding and sampling, where RL fits, and how you evaluate the result.](https://llmdojo.dev/interview/q/cover-the-fundamentals-self-versus-cross-attention-autoregressive-decoding-and-sampling-where)
- [Design and implement a KV cache for autoregressive decoding in a Transformer: tensor shapes for prefill and per-step decode, per-layer caches, batching, memory limits and beam search.](https://llmdojo.dev/interview/q/design-and-implement-a-kv-cache-for-autoregressive-decoding-in-a-transformer)
- [Design an image generation system end to end: data curation, diffusion versus autoregressive architectures, training objectives, compute planning and evaluation.](https://llmdojo.dev/interview/q/design-an-image-generation-system-end-to-end-data-curation-diffusion-versus)

## Related
- [Logit](https://llmdojo.dev/glossary/logit)
- [LM head](https://llmdojo.dev/glossary/lm-head)
- [Greedy decoding](https://llmdojo.dev/glossary/greedy-decoding)
- [Temperature](https://llmdojo.dev/glossary/temperature)
- [Top-k](https://llmdojo.dev/glossary/top-k)
- [Top-p (nucleus sampling)](https://llmdojo.dev/glossary/top-p)
- [Seed](https://llmdojo.dev/glossary/seed)
- [Logprob](https://llmdojo.dev/glossary/logprob)
- [Glossary of every term](https://llmdojo.dev/glossary)