# ML system design interview questions

> 40 ml system design interview questions for AI and ML engineer roles, each with a model answer.

Canonical: https://llmdojo.dev/interview/topic/ml-system-design
Last updated: 2026-10-23

40 questions: 3 junior, 15 mid-level and 22 senior.

## [You need to copy 500 GB of weights to 1,000 GPU workers over shared 10 Gbps links. What's the lower bound on time, and what would you build?](https://llmdojo.dev/interview/q/you-need-to-copy-500-gb-of-weights-to-1-000-gpu)

Level: Senior

```python
size = 500e9                  # bytes
link = 10e9 / 8               # 1.25e9 bytes per second
t_floor = size / link         # 400 s per receiver
t_naive = 1000 * t_floor      # 4e5 s from one source
```

Each worker must receive 500 GB through its own 10 Gbps link, so **400 seconds** is the floor; serving everyone from one source takes about 4.6 days. To approach the floor, **every receiver also uploads**: split weights into chunks and pipeline them down chains or trees, or swarm BitTorrent-style. Time becomes roughly 400 s plus a few chunk times per hop. Respect topology: one copy crosses each rack uplink, then fans out locally. Checksum chunks and resume on failure.

## [Design an LLM coding assistant end to end: data, model and evaluation.](https://llmdojo.dev/interview/q/design-an-llm-coding-assistant-end-to-end-data-model-and-evaluation)

Level: Senior

Data: permissively licensed code, deduplicated and filtered, with **fill-in-the-middle** training so the model completes inside a file, then instruction and repository-level data. Model: a small fast one for inline completion, a larger one for chat and edits. Serving: build context from the cursor's surroundings, open files and retrieved repo snippets, cache the prefix, and cancel stale requests as the user types.

Evaluation: **execution-based** benchmarks like HumanEval and repo-level tasks offline; acceptance rate and retained code online, through A/B tests.

## [Design a job scheduler for ML training and batch inference on a shared GPU cluster.](https://llmdojo.dev/interview/q/design-a-job-scheduler-for-ml-training-and-batch-inference-on-a)

Level: Senior

Jobs enter a queue with a priority and a team **quota**, with fair share across teams. Distributed training needs **gang scheduling**, all GPUs at once or none, and topology-aware placement on NVLink-connected GPUs in the same node or rack.

Batch inference is the filler: preemptible, sharded into small tasks, backfilling idle GPUs. Preemption relies on **checkpointing**, and so does recovery: health checks spot failed GPUs, and jobs restart from their last checkpoint. Track utilisation, queue wait per priority and fairness, and give users a clear view of why a job is waiting.

## [Training a retrieval model over a catalogue of millions of items, you use sampled softmax. Why does it need a correction, and how do you choose the classification loss?](https://llmdojo.dev/interview/q/training-a-retrieval-model-over-a-catalogue-of-millions-of-items-you)

Level: Senior

A full softmax over millions of items is too costly, so each positive competes against sampled negatives. In-batch negatives appear in proportion to popularity, so **popular items get over-penalised**. The **logQ correction** subtracts each item's log sampling probability from its logit, which makes the sampled loss approximate the full softmax.

```python

def in_batch_loss(u, items, log_q):
    # u, items: [B, d]; row i of items is user i's
    # positive; log_q: [B] log sampling probability
    logits = u @ items.T - log_q[None, :]
    labels = torch.arange(len(u), device=u.device)
    return F.cross_entropy(logits, labels)
```

Loss: **softmax cross-entropy** when exactly one item is right among many; per-item binary cross-entropy for multi-label targets or calibrated click probabilities.

## [Design an inference API that converts PDFs into Markdown using several models in stages.](https://llmdojo.dev/interview/q/design-an-inference-api-that-converts-pdfs-into-markdown-using-several-models)

Level: Senior

Split it into stages with different costs: render pages and **detect layout**, then recognise each region with the right model, such as text OCR, tables or formulas, then assemble Markdown in reading order. OCR only pages without a reliable text layer.

Make the API **asynchronous**: upload returns a job id, pages fan out on a queue, and a failed page retries alone. Batch GPU stages across jobs, keep a fast lane for small requests, and cache by file hash. Evaluate each stage, with edit distance to reference Markdown.

## [How can LLMs be incorporated into recommendation systems, from metadata enrichment and cold start to generative retrieval and semantic reranking?](https://llmdojo.dev/interview/q/how-can-llms-be-incorporated-into-recommendation-systems-from-metadata-enrichment-and)

Level: Senior

Use LLMs where they add knowledge ID models lack, off the hot path.

Offline enrichment: generate tags, summaries and embeddings from titles, descriptions and frames. That mostly helps **cold start**, since a new item has meaningful features before it has clicks. **Generative retrieval**: give items semantic IDs, codes from quantized content embeddings, and train a sequence model to generate the next item's ID, as in TIGER. **Semantic reranking**: a distilled LLM rescores the top few dozen candidates against the user's intent. Judge each by offline recall and NDCG, then A/B tests.

## [In the recall stage of a recommender, design a model that chooses how many candidates K to retrieve per request given latency and compute cost.](https://llmdojo.dev/interview/q/in-the-recall-stage-of-a-recommender-design-a-model-that-chooses)

Level: Mid-level

Predict the **marginal value of more candidates** per request. Label offline by replaying traffic at several K values and recording where the final ranker's top results stop changing.

Features: user activity, history length, request context and current load. A small model, say gradient-boosted trees, picks K from a few buckets to maximise **value minus latency and compute cost**, with a safe floor. A/B test against fixed K, watching p99 latency.

## [Design a ride-hailing ETA system, covering labels, features, model choice, low-latency serving, delayed ground truth and drift.](https://llmdojo.dev/interview/q/design-a-ride-hailing-eta-system-covering-labels-features-model-choice-low)

Level: Mid-level

Label: actual trip time. Start from a **routing engine's ETA** over the road graph, then learn a **residual correction** from time of day, live traffic, region, weather and trip type, as DeeprETA does. Serve a compact model with cached features.

Ground truth lands when the trip ends, so track error by segment as labels arrive, watch feature drift meanwhile, and retrain regularly. Pick the loss by business cost, asymmetric if lateness hurts more.

## [How would you build ranking for a food-delivery marketplace, from training labels and objectives to features, architecture and evaluation?](https://llmdojo.dev/interview/q/how-would-you-build-ranking-for-a-food-delivery-marketplace-from-training)

Level: Senior

Two stages: retrieve a few hundred eligible stores, filtered by delivery radius and opening hours, then rank them.

Labels come from the funnel: click, add to cart, order, with orders the main target. A **multi-task model** predicts each step, and the final score blends them with business terms like delivery time and fees. Features: user history and cuisine affinity, store quality, ETA, price, time and location. Labels carry **position bias**, so log position and correct for it. Evaluate offline with NDCG against orders, then **A/B test** orders per session, gross bookings and retention.

## [Design a real-time card fraud detector with a 0.2% fraud rate, 14-day label delay, a 50 ms p95 latency budget and an explicit cost matrix for false positives and negatives.](https://llmdojo.dev/interview/q/design-a-real-time-card-fraud-detector-with-a-0-2-fraud)

Level: Senior

Score every authorisation inside the **50 ms budget**: precompute velocity features, like recent spend per card, in a streaming feature store, and serve a gradient-boosted model in-process.

Labels lag 14 days, so train only on transactions old enough to be labelled, and monitor with faster signals like customer disputes. With 0.2% fraud, keep probabilities calibrated and **set the threshold from the cost matrix**: decline when the expected fraud loss exceeds the expected cost of blocking a good customer. Add rules, a review queue for the grey zone, and **shadow-test** new models.

## [You have trained a fraud model: how would you deploy it with versioning and safe rollbacks, monitor it after launch, and keep inference latency low?](https://llmdojo.dev/interview/q/you-have-trained-a-fraud-model-how-would-you-deploy-it-with)

Level: Mid-level

Ship model, feature code and threshold as **one versioned artifact** in a registry. Run it in shadow mode, then canary it on a slice of traffic, with one-step **rollback** to the previous version.

Monitor latency, score distribution and feature drift, and precision and recall once labels arrive. For **low latency**: precomputed features, a compact model served in-process or on an optimised runtime, and a rules fallback if the model times out.

## [Design a low-latency store recommender for a delivery app's homepage, covering retrieval, pre-ranking, ranking, geospatial caching and feature serving.](https://llmdojo.dev/interview/q/design-a-low-latency-store-recommender-for-a-delivery-app-s-homepage)

Level: Senior

A cascade, cheap to expensive. **Retrieval**: stores that deliver to the user's location, from a geohash index, merged with two-tower candidates and popular-nearby lists. **Pre-ranking** trims them with a light model. **Ranking** runs a richer multi-task model over user, store and context features.

Cache store lists and slow-changing features per geohash cell; ETA and store busyness come from a low-latency feature store. Give each stage a latency budget with timeouts, and fall back to the cached popular list.

## [Target a free-delivery promotion at 20% of customers to maximize incremental orders per dollar: define the uplift estimand, the experiment to collect data and how you evaluate uplift models offline.](https://llmdojo.dev/interview/q/target-a-free-delivery-promotion-at-20-of-customers-to-maximize-incremental)

Level: Senior

The estimand is the **conditional average treatment effect**: the extra orders free delivery causes for a customer, divided by its cost.

Collect data with a **randomised experiment**, treating a random slice, so treatment is independent of features. Fit a T-learner, X-learner or uplift trees. You never see one customer's effect, so evaluate offline with **Qini or uplift curves**: rank holdout customers by predicted uplift and compare treated against control order rates in the top 20%, per dollar spent. Confirm with a live test of the targeted policy against random targeting.

## [Design the architecture of a real-time game recommender for millions of games and tens of millions of users, with request flow and latency SLOs.](https://llmdojo.dev/interview/q/design-the-architecture-of-a-real-time-game-recommender-for-millions-of)

Level: Senior

The client calls a recommendation service, which fetches user features and recent sessions, then **retrieves** candidates from several sources: a two-tower ANN index over millions of games, trending games, and friends' games. It **ranks** a few hundred with a multi-task model and applies **filters and diversity** before returning.

Set an end-to-end latency SLO and split it by stage, with timeouts and a cached fallback list. Stream play events into the feature store so recent sessions count within minutes, retrain daily, and refresh the item index as games launch.

## [Design personalized discovery recommendations for an AI answer engine, from event logging and candidate retrieval to ranking and feedback loops.](https://llmdojo.dev/interview/q/design-personalized-discovery-recommendations-for-an-ai-answer-engine-from-event-logging)

Level: Mid-level

Log impressions, clicks, dwell, follow-up questions and shares, with position, so you know what was shown and ignored. Retrieve candidates from embeddings of the user's past queries, trending stories and editorial picks, and rank with a model predicting **engaged reads**, not clicks alone. Add freshness and diversity rules.

Guard against **feedback loops**: reserve a little traffic for exploration and correct for position. Evaluate offline on time-split logs, then A/B test retention.

## [Design a real-time system that picks and orders a personalized 10-track playlist from 10,000 candidates, with sequence-aware reranking and diversity constraints.](https://llmdojo.dev/interview/q/design-a-real-time-system-that-picks-and-orders-a-personalized-10)

Level: Mid-level

Score all 10,000 candidates with a fast pointwise model of user, context and track, keep the top hundred or so. Then build the list **slot by slot**: a sequence-aware model scores the remaining tracks given the ones already chosen, for flow in tempo, energy and genre.

Apply **diversity constraints** greedily, MMR-style, such as at most two tracks per artist. Evaluate offline on skips and completions, then A/B test **listening time and skip rate**.

## [Design a recommendation system for augmented-reality lenses in a camera app, including cold start for new lenses and new users.](https://llmdojo.dev/interview/q/design-a-recommendation-system-for-augmented-reality-lenses-in-a-camera-app)

Level: Mid-level

Two stages. **Candidate generation** from several sources: lenses near the user's history, trending lenses, lenses friends used, camera context. Then a **ranker** predicting try, capture and send. New lenses: content features like category, creator and a visual embedding, plus a small exploration slot. New users: regional popularity, adapting within the first sessions. Evaluate offline on a time split, then A/B test on **sends, not just tries**.

## [Design a real-time system that shows the top-K points of interest in a user's map viewport as they pan and zoom, within a 150 ms p95 budget.](https://llmdojo.dev/interview/q/design-a-real-time-system-that-shows-the-top-k-points-of)

Level: Senior

Split it into **spatial retrieval** and ranking, both cheap. Index POIs in a spatial grid such as geohash or S2 cells, and **precompute the top candidates per cell and zoom level** offline from popularity and quality, so a viewport query just unions a few cells' lists. Online, a light model reranks a few hundred candidates with user and time features.

To hold 150 ms p95: cache cell lists at the edge and on the client, **prefetch neighbouring cells** while the user pans, and cancel stale requests. If the ranker runs late, serve the precomputed order.

## [Design and build a candidate search system that ranks people from noisy profiles and free-text queries while enforcing hard criteria.](https://llmdojo.dev/interview/q/design-and-build-a-candidate-search-system-that-ranks-people-from-noisy)

Level: Mid-level

Separate **hard filters from soft ranking**. Parse profiles and the query into structured fields, like location, experience and skills, with an LLM extractor plus rules, and apply hard criteria as filters so no ranking can break them. Retrieve with hybrid search over profile text, then rank on skill match, seniority and recency. Show **why each person matched**. Evaluate on recruiter-labelled queries with NDCG, and count hard-criteria violations, which must be zero.

## [What role should LLMs play in fraud detection compared with traditional supervised models, considering data types, accuracy, latency, cost and interpretability?](https://llmdojo.dev/interview/q/what-role-should-llms-play-in-fraud-detection-compared-with-traditional-supervised)

Level: Senior

LLMs aren't the core scorer. Fraud signals are mostly **tabular and behavioural**: amounts, velocities, device and graph features. Gradient-boosted trees or deep models trained on labels do better there, score in milliseconds at tiny cost, and give stable feature attributions for explanations.

LLMs earn a place **around the model**: turning unstructured text such as dispute notes, merchant descriptions and chats into features, summarising cases for investigators, drafting reason codes, and drafting rules. Keep them **off the authorization path**, validate their features like any other, and guard against prompt injection in user-written text.

## [Design a real-time account-takeover model inside payment authorization, handling delayed noisy labels, feature freshness and a tight latency budget.](https://llmdojo.dev/interview/q/design-a-real-time-account-takeover-model-inside-payment-authorization-handling-delayed)

Level: Senior

Score at login and payment within tens of milliseconds. Features: device fingerprint, session behaviour, recent credential or address changes, and velocities, served from a **streaming feature store** and logged at decision time so training sees exactly what serving saw.

Labels arrive late and noisy, from chargebacks and customer reports, so train on **matured windows** plus faster proxies like failed step-up challenges. Rather than only block, pick **actions by risk**: allow, step-up authentication or hold, with thresholds set on expected loss, and a rules fallback on timeout.

## [Set per-user spending limits with reinforcement learning: define the state, actions, transitions and reward, and explain how you'd train and evaluate the policy safely.](https://llmdojo.dev/interview/q/set-per-user-spending-limits-with-reinforcement-learning-define-the-state-actions)

Level: Senior

**State**: the user's risk score, history, balance, recent spend and current limit. **Actions**: a few discrete moves, such as keep, raise or lower by a step. Transitions come from user behaviour. Reward: margin on enabled spend minus expected fraud and credit losses, which arrive late, minus a penalty for declining good spend.

Train **offline from logged decisions** with propensities, using a conservative method, and estimate the policy's value with off-policy estimators such as doubly robust before any live test. Then roll out to a small cohort, inside hard caps set by people.

## [Build a product ranking model for an e-commerce marketplace: what data you collect, how you define labels, which features and model you use, and how you evaluate.](https://llmdojo.dev/interview/q/build-a-product-ranking-model-for-an-e-commerce-marketplace-what-data)

Level: Mid-level

Log impressions with position, plus clicks, add-to-carts and purchases, with query and context. Labels: graded relevance, purchase above add-to-cart above click above skip, corrected for **position bias**, since top slots get clicked regardless. Features: query-product text match, price, ratings, popularity, freshness and user affinity. Model: gradient-boosted trees with a **LambdaMART** objective, then a neural ranker if it pays. Evaluate with **NDCG on later days**, then A/B test on conversion and revenue per search.

## [Design an end-to-end fraud detection model, covering label definition, severe class imbalance, label delay, metrics and the thresholds for action.](https://llmdojo.dev/interview/q/design-an-end-to-end-fraud-detection-model-covering-label-definition-severe)

Level: Mid-level

**Labels**: chargebacks and confirmed fraud are positives, but they arrive weeks later, so train only on orders whose labels have matured. Imbalance: keep every positive, downsample negatives, then re-weight or recalibrate. Features: payment, device, address mismatches and velocities, in gradient-boosted trees. **Metrics**: precision-recall AUC, not accuracy, and recall at your review capacity. **Thresholds come from costs**: approve, review or block, set by expected fraud loss against lost good orders.

## [Design an item category prediction system for a large catalogue, handling hierarchical labels, multimodal features and label quality.](https://llmdojo.dev/interview/q/design-an-item-category-prediction-system-for-a-large-catalogue-handling-hierarchical)

Level: Junior

Predict a path in the taxonomy, like Home, Kitchen, Cookware. Encode the **title, description and image** and combine them. Predict level by level, or the leaf with a loss that respects the tree so mistakes stay nearby. Seller labels are noisy, so **clean them** with agreement checks, and send low-confidence items to review.

## [Sales has limited outreach capacity. Design a lead-scoring model that ranks incoming leads, from labels and features to evaluation against that limit.](https://llmdojo.dev/interview/q/sales-has-limited-outreach-capacity-design-a-lead-scoring-model-that-ranks)

Level: Junior

A lead is positive if it converted within a set window. Features: company data, source and engagement like visits and replies. Train a **calibrated classifier**, such as logistic regression or boosted trees, and rank by probability. If sales can call 100 a week, measure **precision at 100** against the current process.

## [Design a two-stage retrieval and ranking system, built on user embeddings, that surfaces the rental listings a user is most likely to book.](https://llmdojo.dev/interview/q/design-a-two-stage-retrieval-and-ranking-system-built-on-user-embeddings)

Level: Senior

**Candidate retrieval**: a two-tower model embeds the user, from search context and booking and click history, and each listing, from attributes, location and engagement. Train it on bookings as positives with in-batch negatives; an ANN index returns a few thousand listings, filtered to the searched market and available dates.

**Ranking**: a richer model scores those candidates with price, availability and user-listing cross features, predicting booking probability. Cold-start listings rely on content features. Judge offline with NDCG on held-out bookings, then run an **A/B test** on bookings.

## [There is no text query, only structured intent such as location, dates and guests. Design the ranker for home search results, including bias from past rankings, cold start and online evaluation.](https://llmdojo.dev/interview/q/there-is-no-text-query-only-structured-intent-such-as-location-dates)

Level: Senior

Intent is structured: location, dates, guests, filters, plus user history. Rank by predicted **booking probability** from listing, user and context features.

Logged data reflects past rankings, since people click what was shown first, so correct for **position bias**: train with position as a feature and fix it to a constant at serving, or weight examples by inverse propensity estimated from randomised slots. New listings get content-based priors and a little exploration traffic. Evaluate offline with NDCG on bookings, then A/B test bookings and cancellations, watching new-host exposure.

## [Design an ML system that labels listings as family-friendly so guests can filter on it, from labeling strategy to features, evaluation and launch.](https://llmdojo.dev/interview/q/design-an-ml-system-that-labels-listings-as-family-friendly-so-guests)

Level: Senior

Define the label first, in a written policy: amenities like cribs and high chairs, hazards like unfenced pools. Labels come from **host attributes, guest reviews mentioning children, and trained annotators** on a stratified sample, with agreement measured.

Features: amenities, listing text and photos through a multimodal model, review signals. A wrong yes hurts a family more than a missed listing, so tune for **high precision** and send uncertain listings to review. Evaluate per region, launch as an A/B test on filter use, bookings and complaints, and let hosts dispute labels.

## [Design a recommendation system for issues in an issue tracker, covering labels, candidate generation, ranking, real-time serving and permission-aware access.](https://llmdojo.dev/interview/q/design-a-recommendation-system-for-issues-in-an-issue-tracker-covering-labels)

Level: Mid-level

Pick the label first: an issue the user then opens, comments on or picks up. **Candidate generation** pulls from the user's projects, issues linked to recent work, similar issues by embedding, and teammates' activity. A ranker scores them on recency, priority, relation to the user and text similarity. **Filter by permission before ranking**, and refresh cached candidates on events. Evaluate with recall and NDCG on held-out interactions, then A/B test.

## [Design a system that routes incoming bug reports to the right team, using text and log features, labels from past assignments and a human fallback.](https://llmdojo.dev/interview/q/design-a-system-that-routes-incoming-bug-reports-to-the-right-team)

Level: Mid-level

Labels come from **historical assignments**, but tickets get reassigned, so use the team that finally resolved each one and drop teams that no longer exist. Features: title and description embeddings, stack frames and error signatures from logs, component and reporter. Route automatically only when **confidence clears a threshold**; otherwise a triager picks from the top three, and their choice becomes a label. Track per-team accuracy and reassignment rate.

## [Design a real-time home feed ranker that blends long-term personalization with streaming engagement signals, at low latency.](https://llmdojo.dev/interview/q/design-a-real-time-home-feed-ranker-that-blends-long-term-personalization)

Level: Senior

A **funnel** with a latency budget per stage. Retrieval pulls a few thousand candidates from several sources: ANN over item embeddings using a daily long-term user embedding, plus candidates seeded by the user's last few actions. A light pre-ranker trims them; a **multi-task ranker** predicts save, click and hide, combined into one score; a blender enforces diversity and freshness.

Streaming events update a **recent-action sequence** in the online feature store within seconds; the ranker reads it alongside long-term features. Log features as served, so training matches serving. Judge by A/B tests.

## [You own feed ads ranking. Design an ML system to raise ad click-through rate, from the metrics you'd pick to the model architecture and serving.](https://llmdojo.dev/interview/q/you-own-feed-ads-ranking-design-an-ml-system-to-raise-ad)

Level: Senior

CTR alone is gameable: clickbait raises it. The auction ranks by bid times predicted CTR, so pCTR must be **calibrated**, not just well ordered. Offline: log loss, normalized entropy, AUC and calibration by slice. Online: CTR, revenue, advertiser conversions, and hide rate.

Architecture: ad retrieval by targeting and embeddings, then a deep ranker with **feature crosses** over sparse ID embeddings for user, ad, advertiser and context, with multi-task heads for click and conversion. Correct for **position bias**. Retrain continuously as ads churn, and recalibrate after each model change.

## [Design a ranker for ads launched in the last seven days that handles cold start with safe exploration, then hands off smoothly to the mature model.](https://llmdojo.dev/interview/q/design-a-ranker-for-ads-launched-in-the-last-seven-days-that)

Level: Mid-level

New ads have no ID history, so predict from what generalizes: **creative embeddings**, advertiser, category and targeting, with priors from similar ads. Explore with a **capped traffic budget** and Thompson sampling behind a quality floor.

Hand off by blending both models' scores, shifting weight to the mature model as impressions accumulate. Both must stay **calibrated**, since the auction uses the score directly. Measure later CTR and the revenue cost of exploring.

## [Design a short-video recommender end to end, starting with which labels to optimize: watch time, completion, shares, hides and so on.](https://llmdojo.dev/interview/q/design-a-short-video-recommender-end-to-end-starting-with-which-labels)

Level: Mid-level

No single label works: raw watch time favours long videos, completion favours short ones. So predict **several labels** with a multi-task ranker: watch time normalized by duration, completion, like, share, follow, and hides or fast skips as negatives, then combine them with weights tuned against **long-term retention** in A/B tests.

The pipeline: two-tower retrieval over video embeddings, the ranker, then re-ranking for diversity and freshness. Model **position bias** separately so the ranker learns preference.

## [Design a personalized feed ranker and a natural-language-to-HTML generation system. Cover objectives, cold start, exploration and evaluation for each.](https://llmdojo.dev/interview/q/design-a-personalized-feed-ranker-and-a-natural-language-to-html-generation)

Level: Mid-level

Feed: optimize **meaningful engagement**, like opens and remixes, checked against retention. Retrieve, then rank; cold-start users get content embeddings and popular-by-segment items, plus a small **exploration** slot run as a bandit. Evaluate offline with NDCG, then A/B.

HTML: an LLM with design-system components in context and constrained output. Check that the output parses and renders, then compare **rendered screenshots** against references with a calibrated judge. Online, track how often users keep the result.

## [Design a system that predicts how likely a user is to comment on a post, focusing on label definition, features and model choice.](https://llmdojo.dev/interview/q/design-a-system-that-predicts-how-likely-a-user-is-to-comment)

Level: Mid-level

Label: did the user comment within a fixed window after a **real impression**, one actually viewed, so unseen posts aren't false negatives.

Features, taken **as of impression time**: user history and community affinity; post age, community, text embedding and comment velocity; author affinity; context. Start with gradient-boosted trees, then a **multi-task network** that learns comments alongside clicks and votes, since comments are sparse. Evaluate log loss, calibration and AUC, then comments per user online.

## [Design a feature store that serves both offline training and low-latency online inference, with point-in-time correctness and offline-online consistency.](https://llmdojo.dev/interview/q/design-a-feature-store-that-serves-both-offline-training-and-low-latency)

Level: Senior

**One feature definition**, in versioned code, compiled to both batch and streaming jobs. They write an append-only, timestamped offline store and a key-value online store holding the latest value per entity.

Training uses an **as-of join**: each label gets the latest feature value at or before its event time, never after.

For consistency, log served features and compare them with offline recomputation to catch **skew**. CI tests transforms, schemas and backfills. In production, replicate the online store, give every feature a default and a timeout, and alert on freshness, null rates and drift.

## [Design a video recommendation system for the home feed and up-next, focusing on infrastructure, logging and observability.](https://llmdojo.dev/interview/q/design-a-video-recommendation-system-for-the-home-feed-and-up-next)

Level: Senior

One stack, two contexts: home feed has no seed; up-next adds the current video. Candidates come from ANN over video embeddings, co-watch, subscribed communities and trending, then a GPU ranker reads the online feature store.

**Logging is the backbone**: every impression records request ID, model version, candidates, scores, position and features as served. Client events like watch seconds and skips join by request ID into skew-free training data.

Observability: **per-stage latency SLOs**, candidate counts per source, empty results, feature freshness, score and calibration drift, and per-surface business metrics. Ship via canary with **automatic rollback**.

## [Frame named entity recognition for short e-commerce text like listing titles: which algorithm, what data processing, and which metrics?](https://llmdojo.dev/interview/q/frame-named-entity-recognition-for-short-e-commerce-text-like-listing-titles)

Level: Junior

It's **token classification** with BIO tags for brand, model, size, colour. Titles are short and noisy, so a fine-tuned BERT-style encoder beats a CRF baseline. Normalize text, write labelling guidelines, and bootstrap labels from **structured item attributes**. Measure **entity-level** precision, recall and F1 per type, counting only exact span and type matches.

## Related
- [All interview questions](https://llmdojo.dev/interview)