Overfitting — When Models Memorize Instead of Generalize

Why it matters

Every ML project eventually faces overfitting. Understanding detection and prevention is core practical ML.

The architecture

Detection: monitor training and validation loss separately. Training loss decreases; validation loss plateaus or increases → overfitting.

Symptoms: gap between train and val performance.

How it works end to end

Causes: model capacity exceeds signal in data. Small data + big model = overfitting.

Prevention: more training data, augmentation, dropout, weight decay (L2), early stopping (halt when val loss stops improving), simpler model.

Cross-validation: k-fold cross-val estimates generalization when data is limited.