Why it matters
Top-k is simpler and more predictable than top-p. Some settings work better with fixed k. Understanding both helps pick the right one.
Advertisement
The architecture
Sort logits descending. Take top k (typically 40-100).
Sample from these k (renormalized).
Advertisement
How it works end to end
Fixed count means low-quality tokens can be sampled if distribution is very peaked.
Typical values: k=40 (common), k=100 (more diverse), k=1 (greedy).
Combine with temperature: temperature reshapes distribution, top-k truncates.