The journey begins when the model scans its vocabulary — modern tokenizers hold 100,000 to 200,000+ tokens — and assigns a raw score to every candidate. These scores are logits.

Logits are unnormalized real numbers: positive or negative, no upper limit. A higher logit means the model favors the word, but these are not percentages yet.

Insight: these are raw outputs from the neural network’s final layer — preference, before probability.

Before scores become probabilities, we can scale them with the temperature parameter: divide every logit by it.

Softmax exponentiates every logit (making them all positive) and divides by the total. After softmax, every value sits between 0 and 1, and the sum of all values is exactly 1 — 100%.

First, the vocabulary is sorted from highest to lowest probability. Then top-p (nucleus) sampling sets a cumulative threshold — say P = 0.90 — and keeps only the top tokens whose probabilities sum to it. The tail is discarded before sampling, so the model can never blurt a nonsensical word.

Finally, the model samples from the survivors — weighted by their probabilities. That single sampled token is appended to the text, and the entire pipeline runs again for the next one. The weather today is very… hot.