Why it matters
Context extension enables long-context. Understanding shapes technique.
Advertisement
The architecture
Position interpolation: rescale RoPE.
Fine-tune on long examples.
Advertisement
How it works end to end
Linear PI: naive rescale (quality drop).
NTK: freq-aware scale.
YaRN: refined NTK.
LongRoPE: search-based.