Compression changes a representation
Quantization maps many possible original values onto a smaller set of representable values. This reduces storage for selected tensors while introducing approximation. A useful explanation must distinguish the numerical operation from its consequences for a language model. Rounding error in a vector can be computed exactly, but a model response depends on many interacting layers and operations. The vector error is not an accuracy score.
A four-bit representation has fewer codes than an eight-bit representation. Practical model formats also store scaling information and may partition values into groups. Some preserve sensitive values or operations at higher precision. These choices mean that a nominal bit count does not fully describe the format, its storage overhead or the arithmetic executed by the GPU.
Build a deliberately simple example
The exercise uses a fixed synthetic vector generated from sine values and one adjustable outlier. A symmetric uniform quantizer chooses a scale from the largest magnitude, rounds values to signed integer codes, clips them to the selected range and reconstructs approximate values. It displays the scale, mean absolute error and maximum absolute error. The calculation is deterministic and runs entirely in the browser.
This teaching quantizer uses a symmetric range from minus seven to seven for four-bit mode, leaving one of sixteen possible codes unused. Eight-bit mode similarly uses minus 127 through 127. It is intentionally simpler than production quantizers. No claim is made that it reproduces NF4, GPTQ, AWQ, FP8 or any particular checkpoint converter.
Lab: make the outlier larger
Select four-bit mode and calculate the error. Then switch to eight bits with the same vector. The finer representation generally lowers the rounding error in this example. Increase the outlier magnitude and repeat both settings. Because the scale is selected from the largest magnitude, an outlier can make the quantization interval coarser for ordinary values.
The expected outcome is visible numerical sensitivity to precision and scaling. It is not proof that every eight-bit model is more accurate or faster than every four-bit model. Different models, calibration procedures, group sizes and kernels can change the comparison. Use the lab to formulate a testable question, then evaluate the actual configurations with identical tasks.
Educational calculation on synthetic inputs. No device access, model calls or measured GPU results.
Evaluate the output you actually need
A generic evaluation can use synthetic sequences with explicit classification rules and a fixed output schema. Freeze the held-out set before selecting settings. Measure both task correctness and schema validity. A response can be valid JSON while containing the wrong label, so combine those checks rather than treating formatting as the entire task.
Compare the same model revision where possible. Hold prompt templates, output length limits, decoding settings and evaluation inputs constant. If the higher-precision baseline does not fit locally, report that limitation. Comparing two differently sized models is still useful, but it answers a model-selection question rather than isolating the effect of quantization. Preserve unsuccessful cases instead of removing them from the denominator.
Memory, quality and latency are separate axes
A smaller representation can reduce memory traffic, but the runtime may need to unpack values or convert them during computation. The resulting speed depends on the GPU, implementation and workload. Prompt processing and token-by-token decoding can respond differently. Check the serving engine compatibility documentation before assuming a format uses an optimized execution path.
For measured results, report end-to-end latency, generated token count, warmup treatment and sample size. Calculate output tokens per second from the recorded token count and generation duration, naming the runtime field used. Do not present a synthetic rounding lab as a throughput benchmark. This page provides numerical intuition; the scheduled GPU experiment supplies a separate, explicitly named runtime and observed measurements.
Sources and next steps
vLLM quantization compatibility · PEFT LoRA explanation · QLoRA paper
Follow the Local LLM learning path. Evidence label: explanation and calculated simulation. All inputs are synthetic; no measured training or inference claim is made.