LayerNorm Statistics

Normalize one token across its features

This direct calculator treats the input vector as one token row. It subtracts the arithmetic mean of that row, then divides each centered component by sqrt(population variance + epsilon). Population variance divides the sum of squared centered components by the number of features, not by that count minus one. There is no averaging across tokens or a batch. The second stage multiplies each normalized component by its matching gain gamma and adds its matching bias beta. These editable parameters illustrate an affine transformation; no training, gradients or learned checkpoint is present.

Specify all numerical inputs

Input vector accepts a JSON array of one to thirty-two numbers. Gain gamma and Bias beta each accept either a single JSON number, broadcast to every coordinate, or an array exactly matching the input length. All components and affine values must be finite numbers between minus one thousand and one thousand. Epsilon accepts a positive number between 1e-12 and one, and enters inside the square root. Changing input length preserves your affine arrays: a mismatched length is an explicit error until corrected. Reset defaults restores [-1,0,1], gain one, bias zero and epsilon 0.00001. Calculation is immediate and remains inspectable indefinitely; this mathematical instrument needs no artificial playback timer.

Reproduce the denominator and epsilon effect

At the defaults, input mean is zero and variance is two thirds. The denominator is sqrt(2/3 + 0.00001), and the output is [-1,0,1] divided by that number. Normalized variance is (2/3)/(2/3 + epsilon), slightly below one. Set epsilon to one: the denominator becomes sqrt(5/3), normalized variance becomes 0.4 and the output becomes approximately [-0.774596669,0,0.774596669]. The page reports actual statistics rather than labeling every normalized row as exactly unit variance. Add ten to every input component by entering [9,10,11]; the centered and normalized vectors remain unchanged.

Separate normalization from affine output

Keep epsilon one and enter gain two and bias three. The final mean becomes three; its variance becomes four times 0.4, or 1.6. Replace gain with [0,1,-1] and bias with [0,0,0]: the first output is zero and the third normalized component changes sign. With different gains or biases at each coordinate, simple common-scale mean and variance formulas no longer apply. The Per-feature calculations table exposes every centered value, normalized value and affine result. The Population statistics table shows all three stages so you can verify their relationship independently.

Inspect constants, errors and numerical limits

Enter [2,2,2] with scalar gain one and bias zero: variance and normalized output are zero, while the positive denominator is sqrt(epsilon). A single feature also centers to zero. Set bias to [1,2,3] on the constant three-feature input and the final output becomes exactly that array. Invalid JSON, empty input, an affine length mismatch or epsilon zero clears both tables and the plot, avoiding stale results. Full precision calculation opens the computed arrays and denominator. Both tables and the full-precision region support keyboard focus and scrolling on narrow screens. Each bar panel has its own labeled symmetric scale, so equal bar heights across panels do not imply equal values. JavaScript double precision and rounded table text differ from low-precision hardware arithmetic. This lesson does not compare training quality or predict execution speed.