FP16, BF16, INT8 and INT4 reconstruction

Choose a numerical experiment

Reconstruction comparison

Errors use the FP32 reference. Byte estimate is packed payload + one 8-byte scale for integer tensors. Scroll tables horizontally on narrow screens.

Selected format: per-value evidence

Compare representations on the same numbers

This direct calculation converts your decimal list to FP32 first, then compares FP16, BF16 and two explicitly defined symmetric integer quantizers against that same reference. It computes each code, reconstructed value and error locally. There is no trained model, quality score or inference-speed prediction. FP16 uses one sign bit, five exponent bits and ten stored fraction bits. BF16 instead uses eight exponent bits and seven fraction bits. The two sixteen-bit formats therefore differ in both range and precision.

Choose and edit a reproducible experiment

Example selects a fixed list; Load example copies it into Values and immediately recalculates. Changing the selection alone preserves your current data. Values accepts one through sixty-four finite decimal numbers separated by commas or spaces, including scientific notation and signed zero. A value outside the finite FP32 range is rejected. The complete evidence distinguishes the parsed JavaScript number from the rounded FP32 reference, so decimal parsing error is not silently attributed to FP16 or INT4. Reset defaults restores the weights example, clip magnitude one and the FP16 detail view.

Read the floating-point boundaries

Load Rounding ties and inspect FP16: 1.00048828125 is exactly halfway between adjacent half values and rounds to 1, while 1.00146484375 rounds to 1.001953125. Ties choose the even retained significand. Switch Detail format to BF16: 1.00390625 rounds to 1 and 1.01171875 rounds to 1.015625. Load Range boundaries: FP16 preserves its smallest positive subnormal, approximately 5.96e-8, while half that magnitude ties to zero. The value 65504 remains finite but 65520 rounds to infinity. BF16 can represent a much larger range. The lab preserves subnormals rather than emulating hardware flush-to-zero modes.

Control the integer scale and clipping

Integer clip magnitude sets a positive bound C. INT8 uses integer codes from minus 127 through plus 127, and INT4 uses minus 7 through plus 7. These are narrow symmetric ranges: the most negative signed code is deliberately unused. The scale is C divided by the maximum positive code. Each FP32 value is clamped to minus C through plus C, divided by that scale, rounded to nearest with even ties, and reconstructed by multiplying the code by the scale. The clipping counter counts inputs outside the bound even if rounding could also change them. Floating-point results are independent of this control.

Reproduce the outlier tradeoff

Load Outlier and keep C at one. The value twenty is clipped to one; smaller values retain a finer integer grid. Fit integer range changes C to the largest absolute FP32 value in the current list, twenty in this case. Clipping disappears, but INT4 now has a step of twenty divided by seven, and all five small values reconstruct as zero. Compare their individual errors before deciding whether the lower total mean squared error serves your application. Fitting an all-zero list uses C equal to one because no nonzero scale can be inferred. Fit does not alter the input array or floating formats.

Inspect exact values and storage assumptions

Detail format selects only which per-element table is visible; the summary always compares all four formats. Hex code displays the sixteen-bit float encoding or the signed integer code in twos-complement bits. The detail table shows the FP32 reference, reconstruction, classification and signed error. Mean squared error averages squared reconstruction differences over the whole list. Maximum error is absolute. A floating overflow makes both aggregate errors undefined, rather than hiding the failed value or averaging only finite entries. Rounded-to-zero counts nonzero references that become zero. Negative zero remains visible in floating encodings and complete evidence.

Keep payload estimates separate from real files

The byte column gives a hypothetical packed tensor payload plus scale storage. Each float uses two bytes, INT8 one byte, and INT4 packs two codes per byte with the last byte padded for an odd count. Each integer tensor additionally uses one eight-byte JavaScript-number scale for this teaching layout. The inspector does not serialize a file or measure browser heap allocation. Real formats may store other scale types, multiple groups, zero points, alignment and metadata. No GGUF, AWQ or GPTQ compatibility is claimed.

Use the result within its limits

All controls compute immediately, so there is no artificial playback timer, pause or speed. The tables and expandable Complete state evidence remain inspectable until you change an input. Invalid input clears the result tables and reports an error; a previous successful result is never shown as a new computation. This experiment quantizes values only. It omits matrix accumulation, activation distributions, calibration, per-channel scaling, memory bandwidth and downstream model accuracy. Numerical reconstruction error alone cannot predict perplexity or a universal percentage of quality loss.

Format references: NumPy floating-point type layouts and NVIDIA BF16 representation.