Senior interview questionAsked at Mistral AI · interview write-up
Design an inference API that converts PDFs into Markdown using several models in stages.
Model answer
Split it into stages with different costs: render pages and detect layout, then recognise each region with the right model, such as text OCR, tables or formulas, then assemble Markdown in reading order. OCR only pages without a reliable text layer.
Make the API asynchronous: upload returns a job id, pages fan out on a queue, and a failed page retries alone. Batch GPU stages across jobs, keep a fast lane for small requests, and cache by file hash. Evaluate each stage, with edit distance to reference Markdown.
Practise the rest
40 ML system design questions, 492 in all, each with a model answer. Free.
More from Mistral AI interview questions.
More ML system design interview questions
- You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?
- Design an LLM coding assistant end to end: data, model and evaluation.
- Design a job scheduler for ML training and batch inference on a shared GPU cluster.
- Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?
- How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?
- In the recall stage of a recommender, design a model that chooses how many candidates K to retrieve per request given latency and compute cost.