Quantization and computed toy perplexity

Configure a four-token predictor

Compare computed predictions

Natural-log loss, identical scored tokens. These are toy results, not pretrained model benchmarks. Scroll tables horizontally on narrow screens.

Inspect the quantized matrix

Account for every scored token

Evaluate an explicit predictor

This experiment is a four-token bigram predictor with vocabulary A, B, C and D. The previous token selects one row of an editable four by four weight matrix. That row contains four logits, one for each possible next token. A softmax turns those logits into a probability distribution. The lab computes a reference result and then repeats the same calculation after quantizing the matrix. These are actual calculations on this tiny fixture, not measurements from a trained language model. There is no model-size control and no claim that a certain bit width preserves a percentage of language quality.

Choose inputs and preserve token accounting

Example selects a fixture without changing your current inputs; Load example copies both its weights and sequence into the editors. Cycle favors the repeating A B C D pattern. Uniform uses sixteen zero weights. Rounding ties repeats one row to make integer rounding easy to inspect. Weights accepts a JSON array with exactly four rows and four finite numeric entries per row, each between minus twenty and twenty. Token sequence accepts two through sixty-five uppercase vocabulary tokens separated by whitespace. The first token provides context only; every later token is scored once. No beginning, ending, padding or unknown token is silently inserted.

Calculate perplexity from token losses

For each transition, the negative natural logarithm of the assigned target probability is the loss in nats. Mean loss is the sum of those losses divided by the number of scored transitions. Perplexity is the exponential of that mean. It is the inverse geometric mean of the target probabilities, not their arithmetic mean and not the mean of separate sequence perplexities. Load Uniform: every target has probability one quarter, every loss is ln(4), and perplexity is exactly four up to floating-point rounding. Changing the sequence cannot change that uniform result. Repeating a complete set of transitions preserves its mean loss.

Set a precise quantization experiment

Clip magnitude C bounds every weight to the interval from minus C to C. Each bit width b uses a narrow symmetric signed code range with maximum Q equal to 2 raised to b minus one, minus one. The scale is C divided by Q. The bounded weight divided by scale is rounded to the nearest integer with even ties, then multiplied by scale to reconstruct a logit. The most negative signed code is unused. Two bits therefore provide three levels. All four quantized predictors are recomputed on the same sequence, while the unquantized reference stays fixed. Fit weight range sets C to the largest absolute input weight, using 0.001 for an all-zero matrix.

Inspect clipping and rounding separately

Load Rounding ties, set Clip magnitude to seven, and select Detail bit width four. The scale is one. Each row of codes must read zero, two, two, zero because half-integer ties choose the even integer. Reduce the clip to one: the weights 1.5 and 2.5 are clipped in each row, so the detail reports eight clipped entries. Fit weight range restores a bound of 2.5 and eliminates clipping. This does not guarantee a lower perplexity on every sequence. Rounding can move a target probability either upward or downward, even when weight reconstruction error increases.

Read both the matrix and token evidence

Detail bit width changes the reconstructed matrix and the per-transition table; it does not alter any computed summary result. Matrix rows show original logits, integer codes and reconstructed logits in vocabulary order. The token table pairs reference and quantized target probabilities and losses for every scored transition. Delta perplexity subtracts the reference value; a negative delta means this particular sequence became more likely. Weight MSE averages sixteen squared reconstruction errors and is independent of the chosen evaluation sequence. Complete state evidence exposes all probability vectors, total losses, codes and full precision numbers. Narrow screens can scroll the tables horizontally with touch or keyboard.

Use a counterexample to universal quality claims

Keep the Cycle weights and compare the default sequence with A A A A A. Only the evaluation sequence changed, so every reconstructed matrix and weight MSE remains identical, yet the token probabilities and perplexities change. At a clip magnitude of 0.001 the quantized logits are nearly equal and their perplexity approaches four on either sequence. The reference remains unchanged. Compare two, three, four and eight bits in the summary instead of assuming monotonic improvement: their grids differ and a finite evaluation sequence does not guarantee a strict ordering. Reset defaults restores Cycle, clip 1.5 and the four-bit detail view.

Respect the scope of the calculation

This direct numerical tool has no playback timer: results remain inspectable until an input changes. Invalid input clears every result table and reports the error rather than retaining a stale score. Arithmetic uses JavaScript double-precision numbers, including softmax and accumulation; the reference is not labeled FP16 or FP32. This fixture has only one-token context, no learned embeddings, no transformer layers, no activation quantization and no training or calibration set. It predicts neither throughput nor memory allocation. Real evaluations must keep model, dataset, tokenizer, context policy and scored-token accounting comparable. A better perplexity on this short artificial sequence is not evidence of better reasoning, safety or general language ability.

Definition and evaluation context: Hugging Face: perplexity of fixed-length models.