Topic 2
BM25, term by term
Okapi BM25, developed by Robertson and colleagues in the 1980s, is a refinement of TF-IDF, and AI Engineering (Ch. 6, "Term-based retrieval") notes that it and its variants are still widely used and are formidable baselines for newer methods. It changes the term-frequency part in two ways:
- Saturation. The fourth mention of a term adds less than the first. A parameter, k1 (1.2 here), sets how quickly it saturates.
- Length normalisation. The book's reason: longer documents are more likely to contain a term, and to contain it more often. BM25 shrinks the term-frequency factor for documents longer than average. A parameter, b (0.75 here), sets how much; b = 0 turns it off.
The lab computes BM25 live on the fourteen help-centre articles from Module 01. Each term in the query adds IDF × a term-frequency factor.
Predict first
Where a BM25 score comes from
Computed live · 14 help articles · k1 = 1.2, b = 0.75
Query terms after lowercasing and dropping stop words, with how many of the 14 articles contain each:
toy · in 2 · IDF 1.792safe · in 2 · IDF 1.792two · in 1 · IDF 2.303year · in 0 · IDF 0.000old · in 0 · IDF 0.000
| # | Article | Matching terms (term × count → points) | Score |
|---|---|---|---|
| 1 | Toy safety: ages 3 and up10 terms (average 12.3) | toy × 2 → 2.600 + safe × 1 → 1.939 | 4.539 |
| 2 | Toy safety: small parts14 terms (average 12.3) · answers the question | toy × 2 → 2.371 + safe × 1 → 1.695 | 4.066 |
| 3 | Two-factor authentication11 terms (average 12.3) | two × 2 → 3.262 | 3.262 |
Now try “my internet keeps cutting out”. The Wi-Fi article matches “keeps” and the slow-speeds article matches “internet”. Both words are in one article each, so both get the same IDF, 2.303. The slow-speeds article is shorter, 13 terms against 15, so with length normalisation it wins, 2.249 to 2.112. Neither score has anything to do with which article answers the question.
Quick check