Loading…

Skip to content

Model bandwidth utilization (MBU)

Also called MBU

The share of peak memory bandwidth inference uses: parameters × bytes per parameter × tokens per second, over the peak, for one sequence.

Learn it in the lessonMeasuring decode and prefill on a rooflineInference and Model Optimization · Where the Time Goes: Memory, Compute and Batching

Taught alongside

Compute-bound
Memory bandwidth-bound
Arithmetic intensity
Roofline
Continuous batching

Home
Learn
Interview
Playground

Try

Jump to a lesson

Loading…

↑ ↓ move↵ openesc close