Why it matters

Understanding beam search explains classical NLP outputs and helps decide when it's appropriate for your task.

Advertisement

The architecture

Beam size k: number of candidates maintained.

At each step, extend each candidate with each of top-k tokens = k² candidates. Keep top k.

Beam search stepK candidatesparallel beamsExtend eachtop-k tokensPrune to khighest prob overallDeterministic; higher probability but less diverse than sampling
Beam search algorithm.
Advertisement

How it works end to end

Length normalization: raw log-probability sums favor short sequences. Normalize by length.

Diverse beam search: penalize similar beams to increase diversity.

Used heavily in machine translation, summarization. Less common in modern chat where sampling is preferred.