Logit
Also called logits
A raw score the model gives each vocabulary token for being next. Softmax turns logits into probabilities.
Interview questions about Logit
- What are logits, and how does a model go from them to a whole sentence of output?
Logits are the raw scores the LM head gives every vocabulary token, 50,257 for GPT-2, from the last position's vector.
- Given logits from a toy language model, implement greedy, top-k and top-p decoding in NumPy.
Greedy takes the argmax. Both samplers sort the probabilities, keep a prefix, renormalise it and sample.
- Your LM head is too big for one device. How do you shard it by vocabulary or by hidden width, combine the logits, and compute an exact top-k across shards?
Vocabulary sharding: each device holds a slice of rows and computes logits for its slice only.
- Implement focal loss for class-imbalanced binary classification from basic tensor operations, keeping it numerically stable in logit space.
Focal loss scales cross-entropy by (1 − pt) to the power gamma, so well-classified examples add little and training focuses on hard ones; alpha rebalances the…
- With no starter code, implement a GPT-style transformer decoder from scratch in about 30 minutes.
- Write pseudocode for single-head scaled dot-product self-attention with masks and tensor shapes, and for the cross-entropy loss.
State shapes as you go: x is T by d, scores T by T. Mask before the softmax, and normalise stably.
- Explain temperature, top-k and top-p. Why would you usually reach for top-p rather than a fixed k?
Temperature divides every logit by T before softmax: low T sharpens, high T flattens, and T near 0 approaches greedy.
- Turn an LLM into a binary classifier using its log-probs and log-sum-exp.
Don't parse generated text; read the next-token distribution after the prompt.