Why it matters

Repetition kills user experience. Understanding these knobs and setting them right avoids embarrassing bugs.

Advertisement

The architecture

Repetition penalty: divide logit of tokens that already appeared by penalty (default 1.0-1.2). Reduces their probability.

Frequency penalty: subtract token frequency × penalty from logit. Scales with count.

Penalty typesRepetition penaltydivide logitFrequency penaltysubtract by countPresence penaltysubtract if seenModern models often need penalty ~1.1 for repetition; 0 for factual output
Penalty mechanisms.
Advertisement

How it works end to end

Presence penalty: constant penalty for any token already seen. Simpler; doesn't scale with count.

Trade-off: too high penalty makes model avoid common words unnaturally.

Task dependence: factual outputs often need low penalty (allow legitimate repetition of terms). Creative writing benefits from more penalty.