DECODE · INSPECT · EXPLAIN
Repetition penalty: inspect signed-logit changes
Apply a sign-aware penalty only to tokens already present in the context.
Your experiment
Compare penalty 1 and 2 with the same seed. A positive seen-token score is divided by the penalty; a negative score is multiplied. Unseen token scores must be unchanged.
Press Play for a full minute at 1×, or use 0.5× for two minutes. Pause to read a stage. Next stage and the timeline reveal the same computed history; changing any model input resets the inspection without autoplay.
What the calculation means
This is one explicit repetition-penalty convention, applied before temperature and candidate filtering. A token is penalized once for being present, not once per occurrence.
The timeline is teaching time, not measured inference latency. A five-stage inspection is allocated to each generated token, with the complete trace spread over 60 seconds.
Visible arithmetic
| Token | Logit | Penalized | Scaled | Softmax | Filtered |
|---|
Context and cache inspector
Offline teaching fixture, not a trained transformer. An explicit eight-token score table uses only the last context token; unknown last tokens use the “the” row. Prompt tokens are whitespace-separated, not BPE tokens. Greedy or fixed-seed sampling operates on exact visible probabilities. EOS or the selected token limit ends decoding. Cache dimensions are accounting inputs only: no attention matrices or real model weights are executed. Changing cache settings must not change this fixture’s output tokens. Cache mechanics · Decoding reference · Official AICassindra articles.
Inspect presence rather than frequency
This experiment applies a sign-aware repetition penalty to an explicit eight-token score table. It does not run a trained language model. Prompt fixture splits text on whitespace and matches tokens without case sensitivity. The last token chooses a score row; an unknown last token uses the documented the row. Every vocabulary token already present anywhere in the context is penalized once. Repeating its spelling again does not compound the penalty. This convention is different from subtracting a frequency penalty or banning repeated phrases.
Reproduce both signs and the neutral boundary
Set Prompt fixture to sat the cat, Repetition penalty to three, Temperature to one, Top-k candidates to eight and Nucleus top-p to one. At the first stage the cat row assigns sat a raw score of three, the a score of minus one, and on a score of one. The Penalized column must show one for sat, minus three for the, and unchanged one for unseen on. A positive seen score is divided by the penalty; a negative seen score is multiplied. Zero stays zero. Change the prompt to sat sat the cat: the transformed first-step scores remain the same. Penalty one is the neutral boundary. A larger penalty lowers seen scores but does not forbid selecting those tokens, and it can distort useful repetitions.
Separate transformations from selection
Temperature divides the transformed scores before stable softmax. Smaller positive temperatures sharpen the distribution and larger temperatures flatten it. Top-k candidates keeps one through eight highest-ranked entries. Nucleus top-p then retains the smallest ranked prefix reaching its threshold after renormalization within Top-k, including the crossing candidate. Strategy chooses either greedy argmax or a fixed-seed draw from the final probabilities. Sampling seed changes that draw without changing the score table; greedy output ignores the draw. Compare seeds only while holding every other setting fixed. With Top-k one, greedy and sampled selection agree for any seed. The numerical table exposes raw, penalized and scaled scores plus unfiltered and final probabilities.
Follow evolving context rather than one repeated image
Maximum new tokens limits generation to one through sixteen appended tokens. The explicit end-of-sequence token can stop it earlier. Each selected token is appended before calculating the next distribution, so newly generated tokens also enter the seen-token set. Set prompt on, greedy strategy, penalty one and an eight-token limit: the continuation is mat, period, then end-of-sequence. Set a one-token limit and only mat is appended. Retain K/V between decode steps changes accounting, not the output probabilities or tokens: this toy executes no attention. Its inherited byte fields use a fixed twelve-layer, four-KV-head, sixty-four-coordinate, two-byte fixture with batch size one.
Use playback and numerical evidence
Play reveals five stages for each generated token across sixty teaching seconds, regardless of the number of tokens. Pause freezes inspection. Next stage advances one fifth of a token slot, so its time increment changes with the generated sequence length. Inspection seeks directly through the computed trace. Restart returns to zero while preserving parameters and speed. Speed offers quarter, half, normal and double progression; even the fastest full run lasts thirty seconds. Parameter edits reset and pause the trace. Empty prompts or more than thirty-two whitespace tokens clear results and disable playback. The prompt field is limited to 256 characters. Read the signed numerical table rather than inferring signs from bar widths, which represent magnitudes. On a phone, focus and scroll the table horizontally to inspect both probability columns. This is a deterministic teaching fixture, not measured model latency, a tokenizer implementation or evidence of improved writing quality.