Senior interview questionAsked at Google · interview write-up
How does pipeline parallelism interact with GRPO rollouts, policy versions, throughput and staleness?
Model answer
Pipeline parallelism splits layers into stages and streams microbatches through them, leaving a fill-and-drain bubble. A GRPO step generates a group of rollouts per prompt, then trains on group-normalised rewards. Generation is decode, which pipelines poorly, so it often runs on a separate inference engine.
Overlapping next-step rollouts with this step's training keeps GPUs busy, but those rollouts come from an older policy version. Tag each rollout with its version, cap the lag at a step or two, and rely on the loss's importance ratio and clipping.
Practise the rest
10 MoE & parallelism questions, 492 in all, each with a model answer. Free.
More from Google interview questions.
More MoE & parallelism interview questions
- Explain mixture of experts, and what expert parallelism is.
- Compare tensor parallelism and pipeline parallelism.
- Which collective operations does distributed training rely on, and where does each show up?
- Estimate the GPU memory a very large matmul needs, then compare pipeline and tensor parallelism for it.
- Design a pretraining pipeline for a mixture-of-experts model that scales.
- Cover quantization, pruning, distillation and kernel fusion, then compare tensor and pipeline parallelism.