DECODE · INSPECT · EXPLAIN
Stable softmax and temperature
Inspect exponentiation and normalization without overflow-prone direct exponentials.
Your experiment
Keep filtering disabled and compare the relative probabilities at temperature 0.5 and 2. The ordering stays the same; the probability ratios change. Verify the normalized sum.
Press Play for a full minute at 1×, or use 0.5× for two minutes. Pause to read a stage. Next stage and the timeline reveal the same computed history; changing any model input resets the inspection without autoplay.
What the calculation means
The computation uses exp(zᵢ/T − max(z/T)) divided by the sum over vocabulary entries. Values use JavaScript double-precision arithmetic over the visible fixture; displayed probabilities are rounded.
The timeline is teaching time, not measured inference latency. A five-stage inspection is allocated to each generated token, with the complete trace spread over 60 seconds.
Visible arithmetic
| Token | Logit | Penalized | Scaled | Softmax | Filtered |
|---|
Context and cache inspector
Offline teaching fixture, not a trained transformer. An explicit eight-token score table uses only the last context token; unknown last tokens use the “the” row. Prompt tokens are whitespace-separated, not BPE tokens. Greedy or fixed-seed sampling operates on exact visible probabilities. EOS or the selected token limit ends decoding. Cache dimensions are accounting inputs only: no attention matrices or real model weights are executed. Changing cache settings must not change this fixture’s output tokens. Cache mechanics · Decoding reference · Official AICassindra articles.
Inspect a concrete eight-token distribution
This experiment performs one next-token decision over an explicit eight-token vocabulary. Prompt fixture is whitespace-separated text, not a production tokenizer. Only the final token chooses the score-table row; earlier prompt tokens matter only to the repetition penalty. The default final token cat supplies logits [-1,0.4,3,1,-0.5,-1,-3,0] in the displayed vocabulary order. Known rows exist for the, cat, sat, on, mat and the period; an unknown final token deliberately uses the the row. These scores are illustrative constants, not logits from a trained model.
Temperature changes ratios before selection
Temperature divides the penalized logits before exponentiation. Subtracting the largest scaled logit from every scaled value prevents exponential overflow for these fixtures and leaves normalized probabilities unchanged. The sum of unfiltered probabilities is one within floating-point error. Keep penalty one, Top-k eight and Nucleus top-p one. With prompt cat, the sat score is three and on is one, so their probability ratio is exp(2/T). At temperature 0.5 that ratio is exp(4), about 54.59815; at temperature two it is exp(1), about 2.71828. Positive temperature preserves score ordering while changing concentration. The table rounds probabilities to six decimal places, so its displayed sum can differ slightly from one.
Filters and penalties are separate operations
Top-k candidates retains the highest-scoring k entries, from one to eight. Nucleus top-p then keeps the smallest ranked prefix whose cumulative probability reaches its threshold after renormalization within those top-k entries. It includes the token that crosses the threshold. This declared order matters: top-p is not applied independently to the original distribution. Repetition penalty affects tokens appearing anywhere in the prompt, case-insensitively. A positive seen logit is divided by the penalty; a negative seen logit is multiplied by it. An unseen token is unchanged. Enter sat cat and compare penalty one with three: sat is seen and its adjusted score falls from three to one, while the unseen on score remains one. This is a score convention, not a guarantee that repetition disappears.
Selection, seeds and reproducible boundaries
Strategy selects greedy argmax or seeded sampling from the final filtered probabilities. Greedy ignores the random draw and breaks equal scores by vocabulary order. Sampling seed chooses a reproducible pseudorandom draw; it does not change the table probabilities. On this single-token page there is no growing generation history. With Top-k one, both strategies always select the same highest-ranked candidate regardless of seed. A larger temperature can change a sampled token but cannot change an unpenalized greedy argmax by itself. Empty prompts and prompts beyond thirty-two whitespace tokens are rejected and clear the arithmetic table. Prompt text is treated as data, never executable markup.
Inspect every stage at your own pace
Play advances a sixty-second teaching timeline through context, score transformation, normalization and filtering, selection, then append. Pause freezes the current view. Next stage advances one of the five stages without autoplay. Inspection seeks directly anywhere from zero to sixty seconds. Restart returns to the first stage while preserving the chosen parameters and speed. Speed offers quarter, half, normal and double speed; even double speed leaves thirty seconds of playback. A model-parameter edit rebuilds at time zero and pauses. The timeline is not measured inference latency. The table exposes the complete calculation even before a stage is highlighted; Context and cache inspector explains which output has been revealed. Its cache accounting is inherited fixture bookkeeping, not evidence of real attention execution. Numerical tables provide the readable alternative to small canvas labels on a phone.