# Memory bandwidth-bound

> Memory bandwidth-bound: Work whose time is set by how fast data moves from memory to the arithmetic units; more memory bandwidth, or fewer bytes to move…

Canonical: https://llmdojo.dev/glossary/bandwidth-bound

Also called bandwidth-bound, memory-bound.

Work whose time is set by how fast data moves from memory to the arithmetic units; more memory bandwidth, or fewer bytes to move, speeds it up.

Learn it in the lesson: [Two kinds of slow](https://llmdojo.dev/learn/inference-optimization/where-the-time-goes/two-kinds-of-slow) (Inference and Model Optimization · Where the Time Goes: Memory, Compute and Batching).

## Interview questions about Memory bandwidth-bound

- [What's the difference between compute-bound and memory bandwidth-bound work, and which one is decode?](https://llmdojo.dev/interview/q/what-s-the-difference-between-compute-bound-and-memory-bandwidth-bound-work)
- [How would you show, on a given machine, that decode is bandwidth-bound and prefill compute-bound?](https://llmdojo.dev/interview/q/how-would-you-show-on-a-given-machine-that-decode-is-bandwidth)
- [Your LLM training and serving is slow. Reason about memory-bound versus compute-bound kernels, the cost of materializing the attention matrix, HFU versus MFU, and which GPU profiling signals you'd read.](https://llmdojo.dev/interview/q/your-llm-training-and-serving-is-slow-reason-about-memory-bound-versus)
- [Explain FlashAttention, the KV cache and RoPE, and the memory and compute trade-offs each addresses.](https://llmdojo.dev/interview/q/explain-flashattention-the-kv-cache-and-rope-and-the-memory-and-compute)
- [Explain how a server like vLLM runs a decoder-only model efficiently.](https://llmdojo.dev/interview/q/explain-how-a-server-like-vllm-runs-a-decoder-only-model-efficiently)
- [How would you optimise large-scale training and inference on GPUs?](https://llmdojo.dev/interview/q/how-would-you-optimise-large-scale-training-and-inference-on-gpus)
- [Explain the optimisations an ML compiler makes, such as kernel fusion, and how they map to the hardware.](https://llmdojo.dev/interview/q/explain-the-optimisations-an-ml-compiler-makes-such-as-kernel-fusion-and)
- [Walk me through how a model goes from framework code to running on a GPU.](https://llmdojo.dev/interview/q/walk-me-through-how-a-model-goes-from-framework-code-to-running)

## Related
- [Compute-bound](https://llmdojo.dev/glossary/compute-bound)
- [Arithmetic intensity](https://llmdojo.dev/glossary/arithmetic-intensity)
- [Roofline](https://llmdojo.dev/glossary/roofline)
- [Model bandwidth utilization (MBU)](https://llmdojo.dev/glossary/mbu)
- [Continuous batching](https://llmdojo.dev/glossary/continuous-batching)
- [Glossary of every term](https://llmdojo.dev/glossary)