Cosine similarity
How closely two vectors point the same way, from −1 to 1, ignoring their lengths. The usual way to compare embeddings.
Interview questions about Cosine similarity
- Your vector database only supports dot-product search, but your embedding model's documentation says to compare vectors with cosine similarity. What do you do, and why does it work?
Normalize every vector, documents and queries alike, to length 1.
- In a notebook, implement embedding retrieval with cosine similarity. How does cosine relate to the dot product, and when would you choose each?
- What is an embedding table, and what does it mean for two embeddings to be similar?
It's a learned matrix with one row per vocabulary entry, and embedding a token just reads its row.