Lesson 2 · 7 topics · 22 min · intermediate

Term-Based and Embedding Retrieval

You will be able to

Compute term frequency, inverse document frequency and a TF-IDF score, and explain why rare terms weigh more.
Explain BM25's term-frequency saturation and length normalisation, and read a BM25 score term by term.
Explain how an inverted index makes term search fast, and why documents sharing no term are never returned.
Show how term tokenization (stop words, splitting, n-grams) creates and destroys matches.
Describe embedding retrieval as nearest-neighbour search over vectors from one embedding model.
Name the main approximate nearest-neighbour index types and their recall, speed and build-time trade-offs.
Compare term-based and embedding retrieval on speed, quality, cost and exact identifiers.

Topics

Your review

Sign in to keep your quick-check answers. The ones you miss come back for review after 1, 3 and 7 days, until they stick.

Sources

AI Engineering · Chip Huyen, 2025
Chip Huyen. O'Reilly. Used: Ch. 6 "RAG and Agents": "Retrieval Algorithms" > "Term-based retrieval", "Embedding-based retrieval", "Comparing retrieval algorithms".
Hands-On Large Language Models · Jay Alammar & Maarten Grootendorst, 2024
Jay Alammar & Maarten Grootendorst. O'Reilly. Used: Ch. 8 "Semantic Search and Retrieval-Augmented Generation": "Dense Retrieval" > "Caveats of dense retrieval", "Nearest neighbor search versus vector databases".

You will be able to

Compute term frequency, inverse document frequency and a TF-IDF score, and explain why rare terms weigh more.

Explain BM25's term-frequency saturation and length normalisation, and read a BM25 score term by term.

Explain how an inverted index makes term search fast, and why documents sharing no term are never returned.

Show how term tokenization (stop words, splitting, n-grams) creates and destroys matches.

Describe embedding retrieval as nearest-neighbour search over vectors from one embedding model.

Name the main approximate nearest-neighbour index types and their recall, speed and build-time trade-offs.

Compare term-based and embedding retrieval on speed, quality, cost and exact identifiers.

Sources

AI Engineering · Chip Huyen, 2025

Chip Huyen. O'Reilly. Used: Ch. 6 "RAG and Agents": "Retrieval Algorithms" > "Term-based retrieval", "Embedding-based retrieval", "Comparing retrieval algorithms".

Hands-On Large Language Models · Jay Alammar & Maarten Grootendorst, 2024

Jay Alammar & Maarten Grootendorst. O'Reilly. Used: Ch. 8 "Semantic Search and Retrieval-Augmented Generation": "Dense Retrieval" > "Caveats of dense retrieval", "Nearest neighbor search versus vector databases".