Why it matters

GD theory shapes ML optimization. Understanding shapes intuition.

Advertisement

The architecture

Convex + L-smooth: O(1/T) rate.

Strongly convex: O(exp(-T)).

Non-convex: stationary points.

GD convergence ratesConvex + smoothO(1/T) rateStrongly convexexp(-T) rateNon-convexstationary pointsAccelerated (Nesterov) improves to O(1/T^2) for smooth convex
GD theory.
Advertisement

How it works end to end

L-smoothness + convexity.

Strong convexity for fast rates.

Nesterov acceleration.

SGD noise analysis.