# Service level objective (SLO)

> Service level objective (SLO): A target a service promises to meet, such as 'first token within 100 ms and at most 10 ms per token after that'.

Canonical: https://llmdojo.dev/glossary/slo

Also called service level objective, SLO, SLOs.

A target a service promises to meet, such as 'first token within 100 ms and at most 10 ms per token after that'.

Learn it in the lesson: [Goodput: counting only what meets the target](https://llmdojo.dev/learn/inference-optimization/measuring-inference/goodput) (Inference and Model Optimization · Measuring Inference: Latency, Throughput and Goodput).

## Interview questions about Service level objective (SLO)

- [Design the architecture of a real-time game recommender for millions of games and tens of millions of users, with request flow and latency SLOs.](https://llmdojo.dev/interview/q/design-the-architecture-of-a-real-time-game-recommender-for-millions-of)
- [Your team wants to raise the batch size to cut serving cost. What happens to users, and which number would you use to decide?](https://llmdojo.dev/interview/q/your-team-wants-to-raise-the-batch-size-to-cut-serving-cost)
- [Design the routing and scheduling layer in front of an inference fleet that serves many tenants.](https://llmdojo.dev/interview/q/design-the-routing-and-scheduling-layer-in-front-of-an-inference-fleet)
- [Design a production ML inference pipeline covering request routing to models, artifact versioning and deployment, and online feature retrieval.](https://llmdojo.dev/interview/q/design-a-production-ml-inference-pipeline-covering-request-routing-to-models-artifact)
- [(Illustrative numbers.) Model A costs $0.004 per request and is correct 50% of the time; model B costs $0.006 and is correct 95% of the time. Which is cheaper, and what should you track?](https://llmdojo.dev/interview/q/illustrative-numbers-model-a-costs-0-004-per-request-and-is-correct)
- [Design optimised inference pipelines, and tell me how you would benchmark them.](https://llmdojo.dev/interview/q/design-optimised-inference-pipelines-and-tell-me-how-you-would-benchmark-them)
- [Design a video recommendation system for the home feed and up-next, focusing on infrastructure, logging and observability.](https://llmdojo.dev/interview/q/design-a-video-recommendation-system-for-the-home-feed-and-up-next)

## Related
- [Time to first token (TTFT)](https://llmdojo.dev/glossary/ttft)
- [Time per output token (TPOT)](https://llmdojo.dev/glossary/tpot)
- [Inter-token latency](https://llmdojo.dev/glossary/itl)
- [Percentile (p50, p99)](https://llmdojo.dev/glossary/percentile)
- [Throughput](https://llmdojo.dev/glossary/throughput)
- [Goodput](https://llmdojo.dev/glossary/goodput)
- [Time to publish](https://llmdojo.dev/glossary/time-to-publish)
- [Glossary of every term](https://llmdojo.dev/glossary)