# Embedding

> Embedding: A vector of numbers that represents a token or a text, arranged so that similar meanings end up close together.

Canonical: https://llmdojo.dev/glossary/embedding

Also called embeddings.

A vector of numbers that represents a token or a text, arranged so that similar meanings end up close together.

Learn it in the lesson: [From token IDs to vectors](https://llmdojo.dev/learn/model-foundations/embeddings/the-embedding-table) (Model Foundations · Embeddings: From Token IDs to Vectors).

## Interview questions about Embedding

- [What is an embedding table, and what does it mean for two embeddings to be similar?](https://llmdojo.dev/interview/q/what-is-an-embedding-table-and-what-does-it-mean-for-two)
- [People say 'embedding' for three different things. What are they, and why does the difference matter?](https://llmdojo.dev/interview/q/people-say-embedding-for-three-different-things-what-are-they-and-why)
- [Your vector database only supports dot-product search, but your embedding model's documentation says to compare vectors with cosine similarity. What do you do, and why does it work?](https://llmdojo.dev/interview/q/your-vector-database-only-supports-dot-product-search-but-your-embedding-model)
- [You're building search over medicine leaflets with a small English embedding model, and you plan to reply 'no answer found' whenever the top cosine score is below 0.5. What risks do you raise before this ships?](https://llmdojo.dev/interview/q/you-re-building-search-over-medicine-leaflets-with-a-small-english-embedding)
- [Your team switches to a newer embedding model with the same 384 dimensions and starts embedding incoming queries with it. Search quality collapses, but nothing errors. Explain what happened and how you'd run the migration.](https://llmdojo.dev/interview/q/your-team-switches-to-a-newer-embedding-model-with-the-same-384)
- [Embedding models return 1,536 or 3,072 numbers per text. What do those sizes mean?](https://llmdojo.dev/interview/q/embedding-models-return-1-536-or-3-072-numbers-per-text-what)
- [An input embedding table is too large for one device. Compare sharding it by vocabulary with sharding it by hidden dimension.](https://llmdojo.dev/interview/q/an-input-embedding-table-is-too-large-for-one-device-compare-sharding)
- [Your assistant's semantic search over user content is slow and expensive. How do you optimise it: embedding choice, ANN indexing and sharding?](https://llmdojo.dev/interview/q/your-assistant-s-semantic-search-over-user-content-is-slow-and-expensive)

## Related
- [Embedding table](https://llmdojo.dev/glossary/embedding-table)
- [Cosine similarity](https://llmdojo.dev/glossary/cosine-similarity)
- [Dot product](https://llmdojo.dev/glossary/dot-product)
- [Contextual embedding](https://llmdojo.dev/glossary/contextual-embedding)
- [Semantic search](https://llmdojo.dev/glossary/semantic-search)
- [Glossary of every term](https://llmdojo.dev/glossary)