CurriculumPractise these
Module 02
Inference and Model Optimization
Prefill versus decode, latency and throughput, quantization, batching, and cost per request.
- Lessons
- 6
- Topics
- 42
- Minutes
- 145
Lessons
- Lesson 1Measuring Inference: Latency, Throughput and Goodput25 min · 7 topics · intermediate
- Lesson 2Where the Time Goes: Memory, Compute and Batching20 min · 6 topics · intermediate
- Lesson 3Precision and Quantization: Fewer Bytes per Weight20 min · 7 topics · intermediate
- Lesson 4Caching Repeated Work: Prompts, Prefixes and Answers20 min · 8 topics · intermediate
- Lesson 5Cost per Request and per Useful Answer20 min · 7 topics · intermediate
- Lesson 6Benchmarking Inference, and the Module Capstone40 min · 7 topics · intermediate
Interview prep
60 questions on this module's topics.