SPECULATIVE SAMPLING · ACCEPT OR RESAMPLE
Speculative Decoding Acceptance
Watch draft tokens accepted with probability min(1, p/q) or replaced from the residual, and check that the output still follows the target model.
Your experiment
One position, six tokens, a fixed target p and a draft q. Start with the exact rule and an overconfident draft, step through the first six trials, then try the close and shifted drafts, the two shortcut rules, more trials and another seed.
Play replays the computed run over sixty inspection seconds; Next event steps, Restart rewinds, Speed changes the pace and Inspection time scrubs. Any input change recomputes the run and rewinds it without autoplay.
Event trace
Computed data
Final metrics
Single-position teaching model. The exact rule is the one used in speculative sampling (Leviathan et al. 2023; Chen et al. 2023); in real decoding it is applied position by position and generation continues from the first rejection. The two shortcut rules are shown only as counterexamples. Draws come from a seeded generator.
The acceptance rule
A target model gives p over six tokens: cat 0.40, dog 0.25, bird 0.15, fish 0.10, mouse 0.06 and horse 0.04. The draft model proposes x from its own q, chosen with Draft distribution q. The exact rule accepts x with probability min(1, p(x)/q(x)); after a rejection it draws from max(0, p - q), renormalized. With the overconfident draft, which gives cat 0.80, cat is accepted half the time because p/q = 0.5, and every other token is always accepted because the draft underweights it. The opening 1,000 trials accept 55.8% against a theoretical 60%, and the output follows p: OUTPUT MATCHES THE TARGET.
Why the output is exact
Each token can come out in two ways: accepted as a draft, with probability min(p, q), or produced by the residual after a rejection. The residual is zero where q already exceeds p, cat here, and positive elsewhere: dog 0.425, bird 0.25, fish 0.175, mouse 0.1 and horse 0.05. The two paths add up to p exactly, whatever q is. The acceptance rate is the sum of min(p, q): 0.6 for the overconfident and the shifted drafts, 0.95 for the close one. Draft quality changes speed, never the output.
Shortcuts that change the answer
Always keep the draft token outputs q itself: a total variation distance of 0.05 from p with the close draft and 0.4 with the overconfident and shifted drafts. Keep only if it is the target argmax, and otherwise emit the argmax, turns sampling into greedy decoding: cat every time, a bias of 0.6 for every draft. Both make generation look fast while changing what the model says.
Trials, seed and noise
Trials and Seed change the measured numbers only. With 200 trials the run accepts 52%; with 2,000 it accepts 57.6%, closer to the theoretical 60%. Seed 9 accepts 59.8% of 1,000. The chart shows the running acceptance settling toward the dashed sum of min(p, q). The rule bias in the result does not depend on the seed; the measured distance in the metrics does, and that part is sampling noise.
Reading the run and its limits
Play, Next event, Restart, Speed and Inspection time step through the setup, the theory, six individual trials with their ratio and uniform draw, a halfway summary, the aggregate and the verdict across sixty inspection seconds; the table lists p, q, min(p, q), the residual, the rule output and the measured output per token. The model treats one position. In real speculative decoding the rule runs at each drafted position in turn and the first rejection ends the round.