# Logit

> Logit: A raw score the model gives each vocabulary token for being next. Softmax turns logits into probabilities.

Canonical: https://llmdojo.dev/glossary/logit

Also called logits.

A raw score the model gives each vocabulary token for being next. Softmax turns logits into probabilities.

Learn it in the lesson: [One score per token](https://llmdojo.dev/learn/model-foundations/sampling/one-score-per-token) (Model Foundations · Next-Token Prediction and Sampling).

## Interview questions about Logit

- [What are logits, and how does a model go from them to a whole sentence of output?](https://llmdojo.dev/interview/q/what-are-logits-and-how-does-a-model-go-from-them-to)
- [Given logits from a toy language model, implement greedy, top-k and top-p decoding in NumPy.](https://llmdojo.dev/interview/q/given-logits-from-a-toy-language-model-implement-greedy-top-k-and)
- [Your LM head is too big for one device. How do you shard it by vocabulary or by hidden width, combine the logits, and compute an exact top-k across shards?](https://llmdojo.dev/interview/q/your-lm-head-is-too-big-for-one-device-how-do-you)
- [Implement focal loss for class-imbalanced binary classification from basic tensor operations, keeping it numerically stable in logit space.](https://llmdojo.dev/interview/q/implement-focal-loss-for-class-imbalanced-binary-classification-from-basic-tensor-operations)
- [With no starter code, implement a GPT-style transformer decoder from scratch in about 30 minutes.](https://llmdojo.dev/interview/q/with-no-starter-code-implement-a-gpt-style-transformer-decoder-from-scratch)
- [Write pseudocode for single-head scaled dot-product self-attention with masks and tensor shapes, and for the cross-entropy loss.](https://llmdojo.dev/interview/q/write-pseudocode-for-single-head-scaled-dot-product-self-attention-with-masks)
- [Explain temperature, top-k and top-p. Why would you usually reach for top-p rather than a fixed k?](https://llmdojo.dev/interview/q/explain-temperature-top-k-and-top-p-why-would-you-usually-reach)
- [Turn an LLM into a binary classifier using its log-probs and log-sum-exp.](https://llmdojo.dev/interview/q/turn-an-llm-into-a-binary-classifier-using-its-log-probs-and)

## Related
- [LM head](https://llmdojo.dev/glossary/lm-head)
- [Autoregressive generation](https://llmdojo.dev/glossary/autoregressive)
- [Greedy decoding](https://llmdojo.dev/glossary/greedy-decoding)
- [Temperature](https://llmdojo.dev/glossary/temperature)
- [Top-k](https://llmdojo.dev/glossary/top-k)
- [Top-p (nucleus sampling)](https://llmdojo.dev/glossary/top-p)
- [Seed](https://llmdojo.dev/glossary/seed)
- [Logprob](https://llmdojo.dev/glossary/logprob)
- [Glossary of every term](https://llmdojo.dev/glossary)