Mid-level interview question
What's the difference between an embedding model and a reranker?
Model answer
An embedding model encodes query and document separately, so document vectors are precomputed and search scales to millions.
Scroll sideways to see the whole diagram
An embedding model encodes the query and each document separately.
Text version
- An embedding model encodes the query and each document separately.
- Relevance is a comparison between two vectors that never saw each other's text.
- Because document vectors don't depend on the query, they're computed once. That's what makes searching a large collection fast.
- A reranker reads the query and one candidate together, as a single input, so every word of one can attend to every word of the other. Hands-On LLMs calls this a cross-encoder.
- It outputs a relevance score. Nothing can be precomputed, so it runs once per candidate at query time: affordable for a shortlist, not for the whole collection.
A reranker is typically a cross-encoder: it reads query and candidate together and scores relevance. More precise, but one model run per candidate at query time, so it only sees a shortlist.
Practise the rest
6 Hybrid & reranking questions, 492 in all, each with a model answer. Free.
More Hybrid & reranking interview questions
- What is hybrid search?
- How does reciprocal rank fusion work, and why use ranks rather than scores?
- When can fusion make retrieval worse?
- How do you size the candidate pool for a reranker?
- A reranker fixes most of your misses but still prefers “safe for children over three” to “not safe for children under three”. What do you conclude?