Why it matters

NTK is theoretical foundation for wide-net analysis. Practical impact modest.

Advertisement

The architecture

Wide net = kernel regression.

Fixed kernel at init.

Analytic tools.

Neural tangent kernelWide netinfinite width limitKernel regressionfixed at initAnalytictraining dynamicsJacot et al. 2018; theoretical foundation for wide-net understanding
Neural tangent kernel.
Advertisement

How it works end to end

Jacot et al. 2018.

Infinite-width limit.

Kernel regression view.