Evidence label: deterministic synthetic model. The lab runs a synthetic next-token model rather than a neural network. At each step a fixed hash of the position and previous token chooses which content tokens are plausible and how confident the step is; a long tail of junk tokens always carries some probability; and a copying mechanism boosts whatever followed the previous token last time, more strongly after each consecutive copy. The lab then applies temperature, top-k, top-p, min-p and a frequency penalty exactly as described in the article, samples dozens of answers, and measures loops, junk and variety. All numbers are calculated from this synthetic model.
Play the lab
How to play
Move temperature, top-p, top-k, min-p and the frequency penalty, then read the tiles: clean samples, the share of junk tokens, samples stuck in a loop, distinct 3-grams across samples (variety) and how many tokens remain eligible after filtering on average. The probability chart shows the effect of your filters on one step, and the sample grid shows the results token by token. Turn on long generations or the fat-tailed model to recreate production incidents, and use the presets to compare greedy, a common default and hot sampling with min-p. Missions unlock in order and reward the cleanest or most varied solution.
What the model assumes
The vocabulary has 120 tokens: 50 content tokens and 70 junk tokens. About 45% of steps are peaked, with one token far above the rest; the others are open-ended, with three to eight plausible tokens of similar logit. Junk logits sit around −6.4, or −4.6 with the fat-tailed model. The copy boost lifts the copied token slightly above the current best and grows with each consecutive copy, up to a cap. A sample loops if any 4-gram appears three times. Outputs are 50 tokens, or 120 with long generations, and 48 samples are drawn per run.
What it cannot tell you
Real vocabularies are a thousand times larger, real tails have structure (near-synonyms, other languages, formatting tokens), and real repetition depends on the prompt and the model in ways this toy copy rule only imitates. The lab cannot tell you good values for your model; it shows the direction and shape of each knob's effect so that you know what to measure on your own prompts.
Read the full explanation
- Sampling Settings That Fix Real Failures: Temperature, Top-p and Min-p (companion article)
- Logprobs and Calibration: Measuring When a Language Model Is Guessing
- Constrained Decoding: How JSON Schema Mode Masks Invalid Tokens
All AI in Production labs · Lab catalog
- Holtzman et al., The Curious Case of Neural Text Degeneration (ICLR 2020)
- Fan, Lewis and Dauphin, Hierarchical Neural Story Generation (top-k sampling, ACL 2018)
- Nguyen et al., Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs
- Keskar et al., CTRL: A Conditional Transformer Language Model for Controllable Generation (repetition penalty)
- Xu et al., Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation
- Hugging Face Transformers: text generation strategies and parameters
Views are my own and do not represent any employer, past or present.