Why it matters
Top-p produces better creative and coherent output than top-k or pure temperature. Understanding it improves generation quality.
Advertisement
The architecture
Sort logits descending. Take smallest prefix whose cumulative softmax probability > p (typically 0.9).
Sample from this prefix (renormalized).
Advertisement
How it works end to end
Adaptive: with confident distribution, only 1-2 tokens considered. With flat distribution, many tokens.
Typical p values: 0.9 (default), 0.95 (more diverse), 1.0 (no filtering).
Combines with temperature: temperature shapes distribution, top-p truncates it.