Topic 2

Fluent, ranked first, and wrong

Time to measure it. GPT-2 small (124M parameters, released in 2019) is small and old. Modern models get far more of these questions right. The mechanism you will see is the same one they run on.

Try this: go through all 14 prompts and keep your own tally before you look at the summary row.

What the 14 prompts show

On these 14 hand-written prompts, GPT-2's top tokenTokenA piece of text from a model's fixed vocabulary: a word, part of a word, a character or a byte sequence. Models read and write tokens, not characters.Where this is taught was the right answer 9 times and wrong 5 times. The five wrong ones:

prompt ends with …                   top token   p        right answer
capital of Australia is the city of  " Sydney"   0.216    " Canberra" (rank 5)
capital of Canada is the city of     " Toronto"  0.1627   " Ottawa"   (rank 3)
capital of Italy is the city of      " Milan"    0.0781   " Rome"     (rank 2)
… chemical symbol of gold is         " Na"       0.1334   " Au"       (rank 47)
2 + 2 =                              " 3"        0.102    " 4"        (rank 5)

Three things to take from it:

• 9 out of 14 is not an accuracy rate. The prompts were written to have a one-word answer, not sampled from anything. Some give half the answer away: "Alexander Graham" is followed by Bell at 0.999, which tests completing a famous name more than knowing who invented the telephone.
• The wrong answers are plausible. Sydney, Toronto and Milan are each their country's best-known city. The model did not produce noise; it produced the kind of answer that usually fits.
• The text sounds equally sure either way. The probability on Sydney was about 1 in 5, but the greedy sentence states it twice with no hedge. Nothing in the generated words tells you the model was unsure.