Topic 7

Module 01 capstone: GPT-2 in your browser

This is the end of Module 01. You have read GPT-2's numbers in tables and labs. Now you run the model yourself, in your own browser, and use everything from the six lessons on prompts you choose.

The capstone tasks

Keep a short notes file as you go. For each task write the prompt, the settings (temperatureTemperatureA number that divides the logits before softmax: below 1 sharpens the distribution, above 1 flattens it, 0 means greedy.Where this is taught, top-kTop-kSampling only among the k most likely tokens, after renormalising their probabilities.Where this is taught, top-pTop-p (nucleus sampling)Sampling only among the smallest set of most-likely tokens whose probabilities add up to at least p.Where this is taught), the numbers you saw on screen, and one or two sentences of explanation.

1. Count tokensTokenA piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not characters.Where this is taught. Type a sentence with a rare word, a number and your own name. How many tokens does the tokens view show, and where did words split? (Lesson 1.)
2. Make greedy loop. With temperature 0 (greedy), find a prompt whose continuation repeats itself. Note the repeated phrase, and compare a repeated token's probability the first and second time it appears. (Lesson 4.)
3. Catch a hallucinationHallucinationFluent, confident output that is false or not supported by the input.Where this is taught. Find a factual prompt where GPT-2's top choice is wrong. Write down the top-5 probabilities and where the right answer ranks, if it appears. Was the wrong answer's probability high or low? (This lesson.)
4. Ground it. Put one true sentence in front of your task 3 prompt. Does the right answer move up? By how much? (This lesson.)
5. Explain a temperature change. Run one prompt at temperature 0 and again at 1 or higher. Find the first token where the two outputs differ. The bars only show the last step, and a new sampled run will not repeat the old one. So paste the prompt plus the tokens both runs share into the prompt box, set New tokens to 1 and temperature to 1, and generate: the bars now show that exact step. Use them to explain which rank each run picked, and with what probability. (Lesson 4.)
6. Explain the whole path. In three sentences, explain how a prompt becomes text, using the words token, embeddingEmbeddingA vector of numbers that represents a token or a text, arranged so that similar meanings end up close together.Where this is taught, attentionAttentionThe mechanism that lets each token take in information from other tokens, weighted by how relevant they score.Where this is taught, logitsLogitA raw score the model gives each vocabulary token for being next. Softmax turns logits into probabilities.Where this is taught and sampling. (Lessons 1 to 5.)

What "done" looks like

• Every task has a prompt, its settings, and numbers copied from the screen, not from this lesson.
• Task 3 names the wrong answer, its probability and the right answer's rank, and says whether the probability gave any warning.
• Task 5 points at one specific step and one specific candidate bar.
• Task 6 uses all five words correctly and in the right order: text → tokens → embeddings → attention layers → logits → sampling → next token, repeated.

Summary

What Module 01 covered, from prompt to text:

• Tokenization. Models read token IDsToken IDThe integer a tokenizer assigns to a token: its position in the vocabulary. IDs only mean something within the vocabulary that produced them.Where this is taught, not text. Byte-pair encoding builds a vocabularyVocabularyThe fixed list of every token a tokenizer knows. Its size is a design choice made before training.Where this is taught of subwordsSubwordA token that can be a whole word or a fragment of one. Common words get one token; rare words are built from several.Where this is taught, so token counts depend on the tokenizerTokenizerThe component that cuts text into tokens and maps each to an integer ID, and turns IDs back into text.Where this is taught and the language. Count tokens before you send them.
• Embeddings. Each token ID selects one learned row of an embedding tableEmbedding tableThe learned matrix with one row per vocabulary entry. Embedding a token ID means reading its row.Where this is taught. Vectors are compared by direction and distance, and similarity tracks topic, not truth.
• Attention. Each token builds a query, key and valueQuery, key and valueThree learned projections of each token's vector. A query is compared with every key to get attention weights; the weights then mix the values.Where this is taught; attention mixes information from earlier tokens (the causal maskCausal maskIn GPT-style models, the rule that a token can only attend to itself and earlier tokens, never later ones.Where this is taught) into each position, across many heads and stacked transformer blocksTransformer blockThe repeated unit of a transformer: attention and a feed-forward network, each wrapped in a layer norm and a residual connection.Where this is taught. Its cost grows with the square of the sequence length.
• Sampling. The last position's vector becomes one logit per vocabulary token; softmaxSoftmaxTurns a list of scores into probabilities: exponentiate each score, then divide by the total so they sum to 1.Where this is taught gives probabilities. Generation is a loop of pick, append, repeat. Greedy, temperature, top-k and top-p decide how to pick.
• Context windowsContext windowThe maximum number of tokens a model can process at once: prompt and output together.Where this is taught and the KV cacheKV cacheStored keys and values of earlier tokens, so each decode step only computes them for the new token. Exact, but costs memory.Where this is taught. The context window is a hard limit on positions, shared by the prompt and the output, and position information tells the model the order of the tokens. Generation splits into one parallel pass over the prompt (prefillPrefillProcessing the whole prompt in one parallel pass before the first output token can be produced.Where this is taught) and one token at a time (decode). The KV cache keeps each layer's earlier keys and values so decode computes only the new token's, at a memory cost that grows with every token.
• Limits. The model is trained for likely text, not true text. It has no lookup, its knowledge is frozen, it copies patterns, and it builds on its own errors. Probability is a signal, not a guarantee. GroundingGroundingGiving the model relevant, trusted text in its context so its answer can be based on that text rather than memory alone.Where this is taught, constrained output, abstention and evaluation reduce hallucination; nothing in this module removes it.

Next

Module 02, Inference and Model Optimization, is in development. It picks up where lesson 5 left off: prefill versus decode, latency and throughputThroughputWork done per second across all requests, usually output tokens per second. Not the same as how fast any one user's reply arrives.Where this is taught, quantizationQuantizationStoring a model's numbers in a format with fewer bits, such as FP16, INT8 or INT4, to cut memory and the bytes each step reads. Strictly it means converting to integers, but in practice it covers any precision reduction.Where this is taught (the reason the browser model's numbers differ), batching and the cost of each request.