Why it matters

Top-k is baseline sampling. Understanding shapes decoding.

Advertisement

The architecture

Sort tokens desc.

Keep top k.

Renormalize + sample.

Top-k sampling flowSort by probdescendingKeep top kfixed countSamplefrom k tokensCombined with top-p common; top-p adapts, top-k caps
Top-k.
Advertisement

How it works end to end

Common k: 40, 50.

Combined with top-p: apply both filters.

Simple + fast.