Why it matters

Prompt lookup gives free speedup on copy tasks. Training-free win.

Advertisement

The architecture

Find n-gram in prompt matching last k tokens.

Speculate next m.

Verify in one forward.

Prompt lookup decodingMatch n-gramin promptSpeculate mnext tokensVerifyone forwardFree 2-3x on summarization + code where output copies input
Prompt lookup decoding.
Advertisement

How it works end to end

Match last-k tokens in prompt.

Speculate next tokens.

Verify with target.