Deep Black Box - Problem 12: the next word is drawn by probability (Chapter 3)

Question 12

The next word is drawn by probability

Your targets: out of 10 runs, the sentence “The largest ocean on Earth is the Pacific Ocean.” comes out 8.0 times or more.
And on top of that, get 2.5 or more different sentences.
Both are set by temperature. These two targets are values this problem chose.

Probability of the next word (cut down to 6 candidates)

Temperature1.00

Turn it down and it gathers on the favourites; turn it up and the tail starts showing

–times / how often the target sentence comes out of 10 runs (calculated)

0target 8.010

–kinds / how many different sentences come out of 10 runs (calculated)

0target 2.510

Both are averages over many repeats of the 10 runs, which is why they come out as decimals.
At temperature 0.65 what you actually get is nearly always two kinds or three (about 70% of the time between them).

Next word	Temperature 0.20	Temp. 1.00	Temp. 2.50
Pacific	1.000	0.736	0.372
Atlantic	0.000	0.100	0.167
Indian	0.000	0.060	0.137
Arctic	0.000	0.045	0.121
Southern	0.000	0.037	0.112
one	0.000	0.022	0.092
Target sentence, out of 10 runs	10.0	5.1	0.7
Different sentences, out of 10 runs	1.0	5.3	9.2

The bottom two rows are calculated by following these 6 candidates all the way through. Leave the temperature at 1.00 and it answers correctly only 5 times out of 10. Take it down to 0.20 and all 10 runs give the very same sentence.

Where this pays off

Turn the temperature down when you want one single fact back, and up when you want several ways of putting it. The temperature in ChatGPT and in the APIs is this knob.