Scalar chain derivative inspector
Compare x[i+1] = scale * slope[i] * x[i] with x[i+1] = x[i] + scale * slope[i] * x[i]. The signed terminal gradient propagates backward through each exact linear factor. Residuals can shrink, grow or cancel.
Leave empty to use Uniform slope. Otherwise supply exactly Depth numbers from -2 to 2; this array overrides Uniform slope.
Rose: plain chain. Blue: residual chain. The vertical axis automatically covers all nonzero magnitudes; exact zeros are omitted, never replaced by a small positive number. Read signs and zero values in the table.
On a phone, focus the table and scroll horizontally to inspect the rightmost gradient columns.
| Layer | Slope | Plain factor | Residual factor | Plain gradient | Residual gradient |
|---|
Full precision calculation
This direct mathematical inspector computes all layers immediately; it is not an animated training run. There are no measured training times or model-quality predictions.
Architecture context: He et al., Identity Mappings in Deep Residual Networks. This scalar linear experiment is narrower than the paper's neural networks.
Define the actual chain
This calculator compares two scalar linear networks of the same depth. A plain layer multiplies its input by branch scale times slope. A residual layer adds that same branch output to its input, giving a multiplier of one plus branch scale times slope. Both networks receive the same user-supplied terminal gradient at the output. The chain rule multiplies this gradient backward through the selected factors. Layer zero is the input and Layer Depth is the output. These are signed derivatives, not vector norms. Real attention, normalization, nonlinear activations and matrix Jacobians are absent.
Control depth, slope and scale independently
Depth accepts an integer from one through 128 and changes how many factors are multiplied. Uniform slope accepts signed values between minus two and two. Branch scale accepts zero through two and scales the branch in both networks, so the comparison does not silently damp only the residual branch. Terminal gradient accepts minus ten through ten and multiplies every reported gradient; changing its sign reverses signs without changing the chart magnitudes. Optional per-layer slopes accepts a JSON array containing exactly Depth numeric entries, ordered from the input toward the output. A nonempty array overrides Uniform slope, which therefore has no effect until the array is cleared. Invalid numbers, missing fields and a mismatched array clear all computed results.
Reproduce growth, decay and cancellation
Reset defaults selects four layers, slope 0.5, scale one and terminal gradient one. The input plain gradient is 0.5 to the fourth power, or 0.0625; the residual input gradient is 1.5 to the fourth power, or 5.0625. Thus an identity skip does not guarantee a gradient near one. Change slope to minus 0.5: the plain gradient still has magnitude 0.0625, alternating sign between layers, while the residual gradient becomes 0.0625 with positive sign. Change slope to minus one: every residual factor becomes zero and every earlier residual gradient is exactly zero, although the output terminal gradient remains one. At slope minus two and depth three the residual input gradient is minus one. The visible signs are essential evidence that a magnitude-only plot cannot supply.
Inspect heterogeneous layers and the identity boundary
Set Depth three and enter [0.5,-1,2] in Optional per-layer slopes with scale one. From the output, residual gradients are one, three, zero and zero when read backward: the middle layer cancels the gradient. The table lists local factors and gradients at every layer so you can locate this cancellation instead of inferring it from a final number. Clear the array and set Branch scale zero. Every residual layer becomes identity and the gradient equals the terminal gradient everywhere. Every plain layer has zero derivative. Set Terminal gradient zero and all gradients become exactly zero in both networks. Reset defaults clears the custom array and restores all numeric controls plus Inspect layer.
Read the chart and its numerical limits
Inspect layer selects a row for a readable signed summary without changing any model input. The chart plots base-ten logarithms of absolute nonzero gradients and automatically spans their full range. Zero has no finite logarithm, so zero points are omitted and segments across them are broken. The table and full-precision calculation remain the authoritative text alternatives. The rose curve is plain and the blue curve is residual. Six-digit scientific notation in the table is rounded; the expandable calculation preserves JavaScript numeric values. Extremely small products can underflow to zero in double precision and are not proof of exact mathematical cancellation. This direct numerical tool has no simulated training clock, optimizer, measured latency or claim about the trainability of a real Transformer.