# Contextual embedding

> Contextual embedding: A token's vector after the transformer layers have mixed in the surrounding text, so the same word gets different vectors in…

Canonical: https://llmdojo.dev/glossary/contextual-embedding

Also called contextual embeddings.

A token's vector after the transformer layers have mixed in the surrounding text, so the same word gets different vectors in different sentences.

Learn it in the lesson: [Static, contextual and text embeddings](https://llmdojo.dev/learn/model-foundations/embeddings/three-kinds-of-embeddings) (Model Foundations · Embeddings: From Token IDs to Vectors).

## Interview questions about Contextual embedding

- [Anthropic reports large gains from contextual retrieval. What exactly did they measure, and what does it cost?](https://llmdojo.dev/interview/q/anthropic-reports-large-gains-from-contextual-retrieval-what-exactly-did-they-measure)

## Related
- [Embedding](https://llmdojo.dev/glossary/embedding)
- [Embedding table](https://llmdojo.dev/glossary/embedding-table)
- [Cosine similarity](https://llmdojo.dev/glossary/cosine-similarity)
- [Dot product](https://llmdojo.dev/glossary/dot-product)
- [Semantic search](https://llmdojo.dev/glossary/semantic-search)
- [Glossary of every term](https://llmdojo.dev/glossary)