# Utilization

> Utilization: The share of a resource's capacity actually in use. A rented machine that sits idle still bills by the hour, so low utilization raises the…

Canonical: https://llmdojo.dev/glossary/utilization

Also called utilisation.

The share of a resource's capacity actually in use. A rented machine that sits idle still bills by the hour, so low utilization raises the cost per request.

Learn it in the lesson: [Idle time costs money](https://llmdojo.dev/learn/inference-optimization/cost-per-request/idle-time) (Inference and Model Optimization · Cost per Request and per Useful Answer).

## Interview questions about Utilization

- [Design request batching for GPU inference, including scheduling, autoscaling and routing.](https://llmdojo.dev/interview/q/design-request-batching-for-gpu-inference-including-scheduling-autoscaling-and-routing)
- [Your training GPUs sit partly idle waiting for data. How do you confirm that, and how would you design a data-preparation pipeline that keeps them fed?](https://llmdojo.dev/interview/q/your-training-gpus-sit-partly-idle-waiting-for-data-how-do-you)
- [Your LLM training and serving is slow. Reason about memory-bound versus compute-bound kernels, the cost of materializing the attention matrix, HFU versus MFU, and which GPU profiling signals you'd read.](https://llmdojo.dev/interview/q/your-llm-training-and-serving-is-slow-reason-about-memory-bound-versus)
- [Design a shared distributed ML training platform with team quotas, coordinated worker placement, versioned data, experiment tracking and checkpoint recovery.](https://llmdojo.dev/interview/q/design-a-shared-distributed-ml-training-platform-with-team-quotas-coordinated-worker)
- [Design a job scheduler for ML training and batch inference on a shared GPU cluster.](https://llmdojo.dev/interview/q/design-a-job-scheduler-for-ml-training-and-batch-inference-on-a)

## Related
- [Cost per request](https://llmdojo.dev/glossary/cost-per-request)
- [Batch API](https://llmdojo.dev/glossary/batch-api)
- [Cost per useful answer](https://llmdojo.dev/glossary/cost-per-useful-answer)
- [Glossary of every term](https://llmdojo.dev/glossary)