Why it matters

Typical sampling shapes creativity. Understanding shapes alternatives.

Advertisement

The architecture

Compute entropy H of distribution.

Filter tokens with |-log p - H| < threshold.

Sample.

Typical sampling flowEntropy Hof distributionTokens near H-log p ~= HSamplefrom typical setMeister et al. 2022; different from top-p; less mainstream
Typical sampling.
Advertisement

How it works end to end

Meister et al. 2022.

Focus on 'typical' outputs.

May avoid very high-prob boring + very low-prob nonsense.