Senior interview questionAsked at TikTok · interview write-up
How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?
Model answer
Use LLMs where they add knowledge ID models lack, off the hot path.
Offline enrichment: generate tags, summaries and embeddings from titles, descriptions and frames. That mostly helps cold start, since a new item has meaningful features before it has clicks. Generative retrieval: give items semantic IDs, codes from quantized content embeddings, and train a sequence model to generate the next item's ID, as in TIGER. Semantic reranking: a distilled LLM rescores the top few dozen candidates against the user's intent. Judge each by offline recall and NDCG, then A/B tests.
Practise the rest
40 ML system design questions, 492 in all, each with a model answer. Free.
More from TikTok interview questions.
More ML system design interview questions
- You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?
- Design an LLM coding assistant end to end: data, model and evaluation.
- Design a job scheduler for ML training and batch inference on a shared GPU cluster.
- Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?
- Design an inference API that converts PDFs into Markdown using several models in stages.
- In the recall stage of a recommender, design a model that chooses how many candidates K to retrieve per request given latency and compute cost.