Senior interview questionAsked at Anthropic · interview write-up
You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?
Model answer
size = 500e9 # bytes
link = 10e9 / 8 # 1.25e9 bytes per second
t_floor = size / link # 400 s per receiver
t_naive = 1000 * t_floor # 4e5 s from one source
Each worker must receive 500 GB through its own 10 Gbps link, so 400 seconds is the floor; serving everyone from one source takes about 4.6 days. To approach the floor, every receiver also uploads: split weights into chunks and pipeline them down chains or trees, or swarm BitTorrent-style. Time becomes roughly 400 s plus a few chunk times per hop. Respect topology: one copy crosses each rack uplink, then fans out locally. Checksum chunks and resume on failure.
Practise the rest
40 ML system design questions, 492 in all, each with a model answer. Free.
More from Anthropic interview questions.
More ML system design interview questions
- Design an LLM coding assistant end to end: data, model and evaluation.
- Design a job scheduler for ML training and batch inference on a shared GPU cluster.
- Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?
- Design an inference API that converts PDFs into Markdown using several models in stages.
- How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?
- In the recall stage of a recommender, design a model that chooses how many candidates K to retrieve per request given latency and compute cost.