Senior interview questionAsked at Reddit · interview write-up
Design a feature store that serves both offline training and low-latency online inference, with point-in-time correctness and offline-online consistency.
Model answer
One feature definition, in versioned code, compiled to both batch and streaming jobs. They write an append-only, timestamped offline store and a key-value online store holding the latest value per entity.
Training uses an as-of join: each label gets the latest feature value at or before its event time, never after.
For consistency, log served features and compare them with offline recomputation to catch skew. CI tests transforms, schemas and backfills. In production, replicate the online store, give every feature a default and a timeout, and alert on freshness, null rates and drift.
Practise the rest
40 ML system design questions, 492 in all, each with a model answer. Free.
More from Reddit interview questions.
More ML system design interview questions
- You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?
- Design an LLM coding assistant end to end: data, model and evaluation.
- Design a job scheduler for ML training and batch inference on a shared GPU cluster.
- Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?
- Design an inference API that converts PDFs into Markdown using several models in stages.
- How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?