# Embeddings interview questions

> 17 embeddings interview questions for AI and ML engineer roles, each with a model answer.

Canonical: https://llmdojo.dev/interview/topic/embeddings
Last updated: 2026-10-23

17 questions: 3 junior, 5 mid-level and 9 senior. Taught in [Embeddings: From Token IDs to Vectors](https://llmdojo.dev/learn/model-foundations/embeddings).

## [What is an embedding table, and what does it mean for two embeddings to be similar?](https://llmdojo.dev/interview/q/what-is-an-embedding-table-and-what-does-it-mean-for-two)

Level: Junior

It's a **learned matrix** with one row per vocabulary entry, and embedding a token just **reads its row**. all-MiniLM-L6-v2's table is 30,522 × 384, about half its parameters.

**Diagram: An embedding lookup**

1. MiniLM's tokenizer lowercases and splits “the river bank”, then adds two special tokens, [CLS] and [SEP].
2. Each token becomes an ID from the vocabulary. “bank” is ID 2924.
3. The embedding table has one row per vocabulary entry and one column per dimension: 30,522 × 384 learned numbers.
4. Embedding “bank” means reading row 2924. No arithmetic, just a lookup. The same ID always reads the same row, whatever the sentence.

Tokens, IDs, table shape and row values measured with sentence-transformers 6.1.0 (all-MiniLM-L6-v2).

No single coordinate means anything; training shapes which vectors land near which. "Similar" means **pointing the same way**, usually measured with cosine similarity.

## [People say 'embedding' for three different things. What are they, and why does the difference matter?](https://llmdojo.dev/interview/q/people-say-embedding-for-three-different-things-what-are-they-and-why)

Level: Junior

Only the last two **change with context**:

- **Static token embedding**: a table row, identical in every sentence: river "bank" and money "bank" score cosine exactly 1.
- **Contextual token embedding**: the transformer's output at one position, so it depends on context.
- **Text embedding**: one vector per text; MiniLM averages its token outputs, then normalizes.

**Diagram: How one sentence becomes one vector**

1. The tokenizer turns the sentence into 9 token IDs, including the special tokens [CLS] and [SEP].
2. Each ID reads its row of the embedding table: 9 static vectors. “bank” gets the same row here as in a sentence about money.
3. Six transformer layers let every token's vector take in the others. The output is still 9 vectors, but now each depends on its context.
4. Search needs one vector per text, so MiniLM averages the 9 token vectors (mean pooling) into one 384-number vector.
5. Finally the vector is scaled to length 1, so cosine similarity and dot product give the same ranking.

Token count, module order (Transformer → mean Pooling → Normalize) and output length measured with sentence-transformers 6.1.0 (all-MiniLM-L6-v2).

## [Your vector database only supports dot-product search, but your embedding model's documentation says to compare vectors with cosine similarity. What do you do, and why does it work?](https://llmdojo.dev/interview/q/your-vector-database-only-supports-dot-product-search-but-your-embedding-model)

Level: Mid-level

**Normalize every vector**, documents and queries alike, to length 1. Cosine is the dot product divided by both lengths, so on unit vectors the two are **the same number**.

Without that, the dot product also **rewards length**, so a long document can outrank a closer short one. Verify rather than assume: for the lesson's sentences, MiniLM's vectors were 4.86 to 6.46 long until its final normalize step.

## [You're building search over medicine leaflets with a small English embedding model, and you plan to reply 'no answer found' whenever the top cosine score is below 0.5. What risks do you raise before this ships?](https://llmdojo.dev/interview/q/you-re-building-search-over-medicine-leaflets-with-a-small-english-embedding)

Level: Senior

Three risks. First, similarity tracks **topic, not truth**: in our MiniLM data, "This medicine is safe for children." and its negation scored 0.936, far above the best real paraphrase pair at 0.645. For medicine, negation needs explicit tests.

Second, **0.5 means nothing universal**. Scores aren't calibrated and there's always a nearest neighbour, so pick the threshold from labelled real queries and recheck it when the model changes.

Third, MiniLM **truncates after 256 word pieces** according to its model card, so chunk long leaflets.

## [Your team switches to a newer embedding model with the same 384 dimensions and starts embedding incoming queries with it. Search quality collapses, but nothing errors. Explain what happened and how you'd run the migration.](https://llmdojo.dev/interview/q/your-team-switches-to-a-newer-embedding-model-with-the-same-384)

Level: Senior

Each model **arranges its own space**, so matching dimensions only means the dot product can be computed. New-model queries against old-model documents give **effectively random scores**, with no error.

Migrate by **re-embedding the whole collection**, cutting queries and documents over together, and re-checking every threshold, since thresholds are model-specific. Even a preprocessing change invalidates stored vectors. Budget for it: generating embeddings costs money, and storage grows with dimension. At 4-byte floats, a million documents is about 1.5 GB at 384 dimensions against 12.3 GB at 3,072.

## [Embedding models return 1,536 or 3,072 numbers per text. What do those sizes mean?](https://llmdojo.dev/interview/q/embedding-models-return-1-536-or-3-072-numbers-per-text-what)

Level: Junior

That's the **vector's dimension**: each text becomes a point with that many coordinates. More dimensions can hold finer distinctions, but every vector costs more **memory and search time**. The numbers mean nothing alone: vectors compare only with others **from the same model**.

## [An input embedding table is too large for one device. Compare sharding it by vocabulary with sharding it by hidden dimension.](https://llmdojo.dev/interview/q/an-input-embedding-table-is-too-large-for-one-device-compare-sharding)

Level: Senior

By vocabulary, each device owns a range of rows, looks up the tokens it owns, writes zeros for the rest, and an **all-reduce sum** assembles full vectors. It pairs with a tied output head sharded the same way, but popular tokens can make one shard hot.

**Diagram: An embedding lookup**

1. MiniLM's tokenizer lowercases and splits “the river bank”, then adds two special tokens, [CLS] and [SEP].
2. Each token becomes an ID from the vocabulary. “bank” is ID 2924.
3. The embedding table has one row per vocabulary entry and one column per dimension: 30,522 × 384 learned numbers.
4. Embedding “bank” means reading row 2924. No arithmetic, just a lookup. The same ID always reads the same row, whatever the sentence.

Tokens, IDs, table shape and row values measured with sentence-transformers 6.1.0 (all-MiniLM-L6-v2).

**By hidden dimension**, every device holds all rows but a slice of the width. Lookups are local, then an **all-gather** concatenates the slices. Both move about batch × tokens × width numbers; choose by what the next layer wants.

## [Your assistant's semantic search over user content is slow and expensive. How do you optimise it: embedding choice, ANN indexing and sharding?](https://llmdojo.dev/interview/q/your-assistant-s-semantic-search-over-user-content-is-slow-and-expensive)

Level: Senior

Embeddings: shortlist models on public retrieval benchmarks, then pick on your own queries; smaller dimensions or int8 vectors cut memory and latency.

**Diagram: How one sentence becomes one vector**

1. The tokenizer turns the sentence into 9 token IDs, including the special tokens [CLS] and [SEP].
2. Each ID reads its row of the embedding table: 9 static vectors. “bank” gets the same row here as in a sentence about money.
3. Six transformer layers let every token's vector take in the others. The output is still 9 vectors, but now each depends on its context.
4. Search needs one vector per text, so MiniLM averages the 9 token vectors (mean pooling) into one 384-number vector.
5. Finally the vector is scaled to length 1, so cosine similarity and dot product give the same ranking.

Token count, module order (Transformer → mean Pooling → Normalize) and output length measured with sentence-transformers 6.1.0 (all-MiniLM-L6-v2).

**Index**: HNSW gives high recall at low latency but holds full vectors in memory; IVF with product quantisation compresses billions of vectors. Tune the **recall-latency trade** with ef or nprobe, then rerank a shortlist exactly.

**Sharding**: partition by user or tenant, since an assistant searches one person's content, which also enforces isolation; otherwise scatter-gather across shards and merge top-k. Add replicas for throughput.

## [Design an API that serves text embeddings and classification, for both interactive and batch requests.](https://llmdojo.dev/interview/q/design-an-api-that-serves-text-embeddings-and-classification-for-both-interactive)

Level: Mid-level

Two endpoints, one encoder: classification is a **light head on the embedding**. Interactive calls are synchronous, with **dynamic batching** to a latency target.

Batch is an **async job**: submit a file, get a job id, poll or take a webhook, read results from storage, trading latency for cost. Every response names the model version, because vectors from different versions aren't comparable.

## [Design a system that returns a meaningful distance between any two employees to power people search and collaboration recommendations.](https://llmdojo.dev/interview/q/design-a-system-that-returns-a-meaningful-distance-between-any-two-employees)

Level: Mid-level

Build a **collaboration graph**: edges from shared documents, meetings, messages, code reviews and the org chart, weighted by frequency and recency. Learn **node embeddings**, with node2vec or a graph neural network, combine them with text embeddings of each person's work, and use cosine distance.

Validate by predicting held-out future collaborations, then by people-search clicks. Respect **permissions**: only use signals the viewer could see.

## [In your recommendation project, did you use embeddings, what counted as a positive label, and how did you retrain without exposure bias or leakage?](https://llmdojo.dev/interview/q/in-your-recommendation-project-did-you-use-embeddings-what-counted-as-a)

Level: Senior

Yes: a **two-tower model** with user and item embeddings, trained with in-batch negatives. A positive should mean value: a click with real dwell time or a save, never a bare impression.

Retrain on a **time-based split**, computing features only from data before each label's timestamp, so nothing leaks from the future. Exposure bias: the logs only hold what the old model chose to show, so keep a small **exploration slice** of randomised items, correct for popularity in sampled negatives, and weight by inverse propensity. Validate on the following day, then A/B test.

## [Compare PCA and autoencoders as ways to fight the curse of dimensionality, and explain how you'd tell which compression helps downstream.](https://llmdojo.dev/interview/q/compare-pca-and-autoencoders-as-ways-to-fight-the-curse-of-dimensionality)

Level: Senior

In high dimensions data gets sparse and distances concentrate, so models need far more data. **PCA** finds the linear projection that keeps the most variance: fast, deterministic, and easy to size from explained variance. An **autoencoder** learns a nonlinear encoder and decoder by minimising reconstruction error. A linear autoencoder with squared loss spans the same subspace as PCA, so it only wins on nonlinear structure with enough data. Judge them on the **downstream task**, not reconstruction: train the same model on each representation, sweep the dimension, and compare held-out accuracy and stability across seeds.

## [In a notebook, implement embedding retrieval with cosine similarity. How does cosine relate to the dot product, and when would you choose each?](https://llmdojo.dev/interview/q/in-a-notebook-implement-embedding-retrieval-with-cosine-similarity-how-does-cosine)

Level: Mid-level

```python

def top_k_cosine(q, docs, k=5, eps=1e-12):
    n = np.linalg.norm(docs, axis=1, keepdims=True)
    d = docs / np.maximum(n, eps)
    qn = q / max(np.linalg.norm(q), eps)
    scores = d @ qn
    idx = np.argsort(-scores)[:k]
    return idx, scores[idx]
```

Cosine is the **dot product of unit-length vectors**, so normalise once at indexing and any dot-product index ranks by cosine. Raw dot product also rewards vector length. Follow **how the embedding model was trained**: cosine for most sentence-embedding models, dot product when the model puts meaningful signal, such as popularity, in the norm. Guard against zero vectors.

## [Design user and visual-item embeddings for retrieval when a user's history can hold 160,000 events. Cover time-correct training data, cold start and freshness.](https://llmdojo.dev/interview/q/design-user-and-visual-item-embeddings-for-retrieval-when-a-user-s)

Level: Mid-level

A two-tower model. The item tower embeds **image and text content**, so new items get vectors immediately. The user tower runs a transformer over recent actions plus pooled summaries of older history, since 160,000 events won't fit attention.

Build each example **as of its timestamp**: only earlier events, split train and test by time. For freshness, refresh user embeddings daily and add recent actions at request time. New users fall back to onboarding interests.

## [Implement a local training pipeline for a text-embedding recommender built from member profiles, item text and click or skip interactions, with leakage-resistant splits and reproducible inference.](https://llmdojo.dev/interview/q/implement-a-local-training-pipeline-for-a-text-embedding-recommender-built-from)

Level: Senior

Join each impression to the profile **as of that timestamp**, label clicks positive and skips negative, then **split by time**: train on the past, validate on the next window, and keep anything fitted, like vocabularies or normalizers, to the training slice.

Model: a two-tower text encoder for member and item, trained contrastively with in-batch negatives plus skips as hard negatives. Evaluate recall at k against a popularity baseline.

For **reproducibility**, pin seeds, model versions and a data-snapshot hash in one config, ship preprocessing with the model, and test that inference reproduces the training embeddings.

## [Design a service that computes and stores embeddings for user-uploaded documents, images and videos. Cover ingestion, chunking, frame sampling, idempotency and multi-tenant privacy.](https://llmdojo.dev/interview/q/design-a-service-that-computes-and-stores-embeddings-for-user-uploaded-documents)

Level: Senior

Uploads land in object storage and enqueue a job. Workers per modality: documents are parsed and chunked by structure; images are embedded directly; videos are **sampled at intervals and scene changes**, with the audio transcribed and embedded as text. Batch on GPUs.

**Idempotency**: key each output by content hash, model version and chunking config, and upsert, so retries never duplicate. Failures go to a dead-letter queue.

Store vectors with tenant and offsets. **Tenant isolation** is enforced server-side: per-tenant namespaces, per-tenant keys, and deletes that propagate to vectors. Model upgrades re-embed into a new index.

## [Design a streaming system that tokenizes arriving text, embeds it, trains a classifier and serves low-latency predictions.](https://llmdojo.dev/interview/q/design-a-streaming-system-that-tokenizes-arriving-text-embeds-it-trains-a)

Level: Senior

Text arrives on a log. Stream workers tokenize with the tokenizer **pinned to the encoder version**, embed in GPU micro-batches, and store vectors with ID, timestamp and version.

**Diagram: How one sentence becomes one vector**

1. The tokenizer turns the sentence into 9 token IDs, including the special tokens [CLS] and [SEP].
2. Each ID reads its row of the embedding table: 9 static vectors. “bank” gets the same row here as in a sentence about money.
3. Six transformer layers let every token's vector take in the others. The output is still 9 vectors, but now each depends on its context.
4. Search needs one vector per text, so MiniLM averages the 9 token vectors (mean pooling) into one 384-number vector.
5. Finally the vector is scaled to length 1, so cosine similarity and dot product give the same ranking.

Token count, module order (Transformer → mean Pooling → Normalize) and output length measured with sentence-transformers 6.1.0 (all-MiniLM-L6-v2).

Train a light head, logistic regression or an MLP, on frozen embeddings, joining labels by time; retrain periodically and promote only after validating on the newest window.

Serving latency is mostly the encoder: use a small distilled one, batch, and cache repeated texts. Keep **training and serving on one embedding version**, and backfill when it changes.

## Related
- [All interview questions](https://llmdojo.dev/interview)