What you're seeing
Gradient descent: x ← x - lr·∇f(x). Small lr: slow convergence. Large lr: oscillation or divergence. Just-right: monotonic descent to minimum.
Modern optimizers (Adam, AdamW): add momentum + per-parameter adaptive scaling. Standard for LLM training and most deep learning.
- Set LR very high (1.0+) — diverges.
- Set LR very low — convergence takes forever.
- Find the sweet spot (~0.1-0.3 for this loss surface).
Follow the exact update
The loss is f(x,y) = x squared plus two times y squared, with its minimum at the origin. Its gradient is (2x,4y). Starting at (2.5,2), each update multiplies x by one minus twice the learning rate and y by one minus four times the learning rate. The contour ellipses are equal-loss curves; the trajectory joins the computed iterates. The green marker is the minimum, the dark gold path shows previous steps, and the red marker identifies the current iterate.
Inspect thirty steps at your pace
LR selects a learning rate from 0.01 to 2.00. Changing it rebuilds the trajectory from the same starting coordinates and pauses at the beginning. Step advances one update. Play reveals thirty updates across sixty seconds at normal speed; Pause holds the current state. Restart rewinds with the current learning rate. Speed changes playback to half, normal or double speed without changing the mathematics. Inspection seconds lets you scrub backward or forward; one update is revealed every two inspection seconds. The text output reports the exact iteration, coordinates and loss even when the point leaves the plot.
Test the stability boundary
Set LR to 0.25 and press Step. The y coordinate becomes zero immediately and x becomes 1.25; later steps halve x. Set LR to 0.50 and restart. The x coordinate becomes zero but y alternates between positive and negative two, so the loss does not approach zero. This boundary case is oscillation without decay, not convergence. Set LR to 0.60 and compare: the y magnitude grows by a factor of 1.4 each update. At 0.10 both coordinates contract and the trajectory approaches the origin.
What this leaves out
This is exact full-gradient descent on a two-variable convex quadratic, not training on a dataset. There is no stochastic sampling, momentum, Adam state, clipping, regularization or floating-point precision comparison. For this starting point convergence requires a positive learning rate below 0.50. The displayed convergence tolerance checks both coordinate magnitudes against 0.01, but the precomputed thirty updates continue so that the timeline remains inspectable. Sixty seconds is teaching time, not optimizer runtime. Large divergent coordinates may leave the fixed plotting window; the numeric output remains the source for those values.