Mid-level interview questionAsked at Shopify · interview write-up
Build a product ranking model for an e-commerce marketplace: what data you collect, how you define labels, which features and model you use, and how you evaluate.
Model answer
Log impressions with position, plus clicks, add-to-carts and purchases, with query and context. Labels: graded relevance, purchase above add-to-cart above click above skip, corrected for position bias, since top slots get clicked regardless. Features: query-product text match, price, ratings, popularity, freshness and user affinity. Model: gradient-boosted trees with a LambdaMART objective, then a neural ranker if it pays. Evaluate with NDCG on later days, then A/B test on conversion and revenue per search.
Practise the rest
40 ML system design questions, 492 in all, each with a model answer. Free.
More from Shopify interview questions.
More ML system design interview questions
- You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?
- Design an LLM coding assistant end to end: data, model and evaluation.
- Design a job scheduler for ML training and batch inference on a shared GPU cluster.
- Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?
- Design an inference API that converts PDFs into Markdown using several models in stages.
- How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?