Why it matters
Backprop is the invisible engine of neural network training. Understanding it demystifies why training works and why some architectures train better than others.
The architecture
Forward pass: compute loss from input by running through network.
Backward pass: compute gradient of loss w.r.t. each weight via chain rule.
Update: adjust weights via optimizer (SGD, Adam) in negative gradient direction.
How it works end to end
Chain rule: gradient of composed function = product of gradients. Backprop propagates gradient from output loss back through layers, multiplying by local Jacobians.
Vanishing gradients: gradient shrinks through many layers. Addressed by ReLU, batch norm, residual connections.
Exploding gradients: gradient grows. Addressed by gradient clipping.
Autograd: modern frameworks track operations and compute gradients automatically.