Why it matters

Landscape shape determines algorithm choice. Understanding shapes ML + optimization intuition.

Advertisement

The architecture

Convex: any local min is global min.

Non-convex: many local minima; saddle points.

Landscape shapesConvexglobal via gradientNon-convexlocal optima trapEscaping saddlesmomentum + noiseDeep learning is non-convex but SGD often finds good solutions
Landscape characteristics.
Advertisement

How it works end to end

Convex: LP, QP, some ML (logistic regression).

Non-convex: neural networks. Saddle points more common than local minima in high dim.

Techniques: momentum + noise escape saddles.