Why it matters

Every model selection involves this tradeoff. Understanding it prevents naive complexity increases and clarifies why regularization works.

Advertisement

The architecture

High bias: model too simple. Underfits. Poor on both train and test.

High variance: model too complex. Overfits. Great on train, poor on test.

Bias-variance curveSimple modelhigh bias, low varianceComplex modellow bias, high varianceSweet spotbalancedAs complexity grows: bias falls, variance rises; total error is U-shaped
Bias-variance dynamics.
Advertisement

How it works end to end

Bias sources: model can't represent true function; too few features; too much regularization.

Variance sources: model capacity exceeds signal; small training data; too few regularization.

Modern picture: deep learning breaks the classic U-curve. Very over-parameterized models generalize well (double descent).

Regularization reduces variance without much bias increase.