# Token

> Token: A piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not…

Canonical: https://llmdojo.dev/glossary/token

Also called tokens.

A piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not characters.

Learn it in the lesson: [Models do not read text](https://llmdojo.dev/learn/model-foundations/tokenization/models-do-not-read-text) (Model Foundations · Tokenization: From Text to Token IDs).

## Interview questions about Token

- [What is a token, and why do most models use subword tokens rather than whole words or single characters?](https://llmdojo.dev/interview/q/what-is-a-token-and-why-do-most-models-use-subword-tokens)
- [A colleague estimates next month's API bill by counting the words in a batch of English and Hindi support tickets and multiplying by the price per token. What's wrong with the estimate, and how would you do it instead?](https://llmdojo.dev/interview/q/a-colleague-estimates-next-month-s-api-bill-by-counting-the-words)
- [Product wants to double the context you support, from 1,000 to 2,000 tokens. From the attention side, what gets more expensive, and what families of fixes would you weigh?](https://llmdojo.dev/interview/q/product-wants-to-double-the-context-you-support-from-1-000-to)
- [A PM wants to show users a 'confidence' percentage computed from the answer's token probabilities. What's your position, and what would you build instead?](https://llmdojo.dev/interview/q/a-pm-wants-to-show-users-a-confidence-percentage-computed-from-the)
- [What is a context window, and why can't GPT-2 small take a 1,025th token even with plenty of memory?](https://llmdojo.dev/interview/q/what-is-a-context-window-and-why-can-t-gpt-2-small)
- [Request A has a 960-token prompt and 10 output tokens. Request B has a 16-token prompt and 200 output tokens. Same model and machine: which finishes first, and what does that tell you about a chat app that resends the full history every turn?](https://llmdojo.dev/interview/q/request-a-has-a-960-token-prompt-and-10-output-tokens-request)
- [Implement causal multi-head attention, add a KV cache for token-by-token decoding, then extend it to grouped-query attention.](https://llmdojo.dev/interview/q/implement-causal-multi-head-attention-add-a-kv-cache-for-token-by)
- [What is a hallucination, and why can a next-token predictor produce one so fluently?](https://llmdojo.dev/interview/q/what-is-a-hallucination-and-why-can-a-next-token-predictor-produce)

## Related
- [Tokenizer](https://llmdojo.dev/glossary/tokenizer)
- [Token ID](https://llmdojo.dev/glossary/token-id)
- [Vocabulary](https://llmdojo.dev/glossary/vocabulary)
- [Byte pair encoding (BPE)](https://llmdojo.dev/glossary/bpe)
- [Subword](https://llmdojo.dev/glossary/subword)
- [Special token](https://llmdojo.dev/glossary/special-token)
- [Glossary of every term](https://llmdojo.dev/glossary)