Why it matters

NTK-aware improves PI. Understanding shapes extension.

Advertisement

The architecture

Scale base b by scale^(dim / (dim - 2)).

High freq preserved; low freq compressed.

NTK-aware scalingBase freq boriginalScale bs^(d/(d-2))High freq intactlow freq compressedKaiokendev's insight; YaRN refined with temperature + attention scaling
NTK-aware.
Advertisement

How it works end to end

Kaiokendev + community discovery.

Better than PI without fine-tune.

YaRN adds attention scaling.