DECODING · CANDIDATE FILTERS
Top-k and nucleus sampling: inspect retained candidates
Filter six next-token distributions, three peaked and three flat, with top-k, top-p or min-p, then draw from what is kept.
Your experiment
Each of six steps has eight candidates with fixture probabilities; some are wrong or meaningless. Start with top-k at k = 5, then search for a k that fits every step, switch to top-p and min-p, combine top-k with top-p, and change the seed.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, Speed changes the pace and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Fixture distributions, not a trained model, and the good or wrong labels are part of the fixture. After filtering the kept probabilities are renormalized; 100 seeded draws per step are taken from them. Top-p keeps the smallest prefix whose probability reaches p; min-p keeps candidates with at least the threshold times the top probability.
Peaked steps and flat steps
Six decoding steps alternate between peaked contexts, where one continuation is obvious (The Eiffel Tower is in: Paris 0.86, France 0.10), and flat contexts, where six continuations are all reasonable (My favourite colour is: blue 0.20 down to yellow 0.135). Every step also has wrong or meaningless candidates in its tail. After filtering, the kept probabilities are renormalized and 100 seeded draws are taken per step, 600 in total.
A fixed k cannot fit both shapes
With top-k and k = 5 every step keeps 5 candidates: each peaked step keeps 3 wrong ones and 8 of the 600 draws are wrong, while each flat step cuts one good choice: WRONG TOKENS AND CUT CHOICES. k = 2 suits the peaked steps but cuts 12 good choices from the flat ones; k = 6 keeps every good choice but also 12 wrong candidates, and 8 draws are wrong. No single k serves both shapes.
Top-p and min-p adapt to the shape
Top-p keeps the smallest set whose probability reaches p. At p = 0.9 it keeps 2 candidates on peaked steps and 6 on flat ones: KEPT SET FITS EVERY STEP. At 0.85 the peaked steps keep only their top token and cut France, four and C; at 1.0 nothing is filtered and 15 of 600 draws are wrong, as with No filter. Min-p keeps candidates with at least the Min-p threshold times the top probability: 0.1 fits every step, 0.06 lets one wrong candidate into each flat step (6 wrong draws) and 0.12 cuts the second good token from the peaked steps. p applies only to top-p and the threshold only to min-p.
Combining filters and changing the seed
Top-k, then top-p applies k first. With k = 5 and p = 0.9 the flat steps lose their sixth good choice, 3 cut in total, although top-p alone would keep it; with k = 6 the combination fits every step. Filters applied in sequence give the stricter answer at each step. Sampling seed changes the draws, not the kept set: with no filter, seeds 1, 2, 5 and 20 draw 19, 18, 15 and 20 wrong tokens.
Reading the run and its limits
Play, Next event, Restart, Speed and Inspection time step through the setup, each of the six steps, the summary and the verdict across sixty inspection seconds; the table lists what each step keeps, the kept mass, wrong candidates kept, good candidates cut and wrong draws. The distributions and their good or wrong labels are fixtures, not a trained model. Real decoders apply temperature and penalties before these filters, and the best settings depend on the model and the task.