Why it matters

PI is naive baseline. Understanding shapes extension theory.

Advertisement

The architecture

Scale positions by L_new / L_old.

All frequencies compressed equally.

Position interpolationOriginal positions0 to L_oldLinear scaleL_new / L_oldCompressed frequenciesquality dropNTK-aware and YaRN improve on PI by preserving high-frequency information
PI.
Advertisement

How it works end to end

Chen et al. 2023.

Divide position by scale factor.

Requires fine-tune.

NTK-aware better.