DECODE · INSPECT · EXPLAIN
From logits to a selected token
Inspect every arithmetic stage between an unnormalized score and a token.
Your experiment
Step through the five stages. Reconcile the score, penalized score, temperature-scaled score, softmax probability and filtered probability columns before revealing the selected token.
Press Play for a full minute at 1×, or use 0.5× for two minutes. Pause to read a stage. Next stage and the timeline reveal the same computed history; changing any model input resets the inspection without autoplay.
What the calculation means
Logits are not probabilities. Subtracting the largest scaled logit before exponentiation improves numerical stability without changing the normalized result.
The timeline is teaching time, not measured inference latency. A five-stage inspection is allocated to each generated token, with the complete trace spread over 60 seconds.
Visible arithmetic
| Token | Logit | Penalized | Scaled | Softmax | Filtered |
|---|
Context and cache inspector
Offline teaching fixture, not a trained transformer. An explicit eight-token score table uses only the last context token; unknown last tokens use the “the” row. Prompt tokens are whitespace-separated, not BPE tokens. Greedy or fixed-seed sampling operates on exact visible probabilities. EOS or the selected token limit ends decoding. Cache dimensions are accounting inputs only: no attention matrices or real model weights are executed. Changing cache settings must not change this fixture’s output tokens. Cache mechanics · Decoding reference · Official AICassindra articles.
Follow one decision from scores to an appended token
This page executes one next-token choice over eight named vocabulary entries: the, cat, sat, on, mat, a period, an end-of-sequence marker and a. Prompt fixture is split on whitespace. Its last token selects a fixed score row, with known rows for the, cat, sat, on, mat and the period. Unknown final tokens deliberately select the the row. These small deterministic scores are supplied teaching data, not output from a pretrained model or an output-projection matrix. Earlier prompt tokens matter only to the repetition penalty. This lesson ends after one new token; it does not continue a conversation.
Transform logits and read the full table
Start with prompt on, penalty one, temperature one, Top-k eight and Nucleus top-p one. The raw score for mat is three and for the is two. Their unfiltered probability ratio is exp(1), approximately 2.71828. Softmax subtracts the largest scaled score, exponentiates and divides by the sum. A negative logit is a score, not a negative probability. Temperature divides every penalized score; temperature two makes that same ratio exp(0.5). The numerical table exposes raw, penalized and scaled logits plus probabilities before and after filtering. Canvas bar widths represent score magnitudes during the early stages; read the table to recover signs.
Separate a decision rule from a distribution
Strategy chooses greedy argmax or seeded sampling. Greedy chooses the largest filtered probability, breaking exact ties by vocabulary order. For the unpenalized on row this is mat regardless of positive temperature. Seeded sampling uses a deterministic pseudorandom draw to select from the final probabilities; the draw changes with Sampling seed, while the probability table does not. Set prompt cat, temperature two, penalty one, Top-k eight and top-p one. In sampled mode seed one selects sat while seed 9999 selects cat. Switch to greedy with those same probabilities and it selects sat. This is reproducibility within the declared fixture, not a guarantee of matching another library random generator.
Control candidate pruning and history effects
Top-k candidates retains the highest-scoring k entries, from one to eight. Nucleus top-p then keeps the smallest ranked prefix reaching its threshold after renormalizing within those k entries; the crossing token is included. Set Top-k one: both strategies select the same highest-ranked token for every seed. Restore eight and use top-p 0.05: only the first candidate remains for these score rows. Repetition penalty divides positive seen-token logits and multiplies negative seen-token logits; unseen tokens are unchanged. Enter sat cat and penalty three: the seen sat score falls from three to one. Penalty one disables this transformation. Presence is case-insensitive and repeated occurrences are not counted repeatedly.
Inspect the stages and respect the scope
Play advances a sixty-second teaching timeline through context, score transformation, normalization and filtering, selection, then append. Pause freezes it; Next stage moves twelve teaching seconds; Inspection seeks directly from zero to sixty. Restart returns to the first stage while retaining parameters and speed. Speed offers quarter, half, normal and double progression; even double speed lasts thirty seconds. A parameter edit rebuilds and pauses at the start. Empty prompts and more than thirty-two whitespace tokens are invalid and clear results. Text resembling markup stays literal. Context and cache inspector exposes the current phase and revealed token. Its inherited cache fields are bookkeeping for a fixed toy architecture, not evidence of attention execution. The timeline is not inference latency. Use the scrollable numerical table as the readable alternative to small phone canvas labels.