Warm-up

Also called warm-up runs

Untimed runs before a benchmark starts timing, so one-off costs such as loading and kernel compilation don't land in the results.

Interview questions about Warm-up

You time a GPU request by reading the clock right after calling the model and get 0.3 ms per token. What went wrong, and what else can silently skew a timing?
GPU calls are asynchronous: they return before the GPU finishes, so I timed the queuing, not the work.
What does a fair inference benchmark hold fixed?
Everything except the thing you changed: model and precision, prompt and reply lengths, batch, concurrency and arrival rate, warm-up, repeats, environment and…