Why it matters

NTK is theoretical foundation. Understanding shapes DL theory.

Advertisement

The architecture

Infinite-width NN linearizes around init.

Kernel: NTK.

Training: kernel regression.

NTK theoryWide networkwidth → infinityLinearizearound initKernel regressionwith NTKExplains lazy learning regime; finite-width NNs differ from NTK
NTK.
Advertisement

How it works end to end

Jacot et al. 2018.

NTK: fixed kernel from network architecture.

Lazy vs feature-learning regimes.

Feature learning: mean-field theory.