| Dimension | Function | Frequency | Wavelength | Value |
|---|
Full precision and all 128 samples
What you're seeing
Sinusoidal PE: PE[pos, 2i] = sin(pos / 10000^(2i/d)), PE[pos, 2i+1] = cos(pos / 10000^(2i/d)).
Each dimension uses a different frequency. Low i: changes fast (handles fine-grained position). High i: changes slowly (handles long-range coarse position).
- Add more dimension indices to dimensions to see multiple frequencies at once.
- Compare dim 0 (fastest) with dim d-1 (slowest).
- RoPE uses the same per-dim frequencies, but rotates Q/K instead of adding.
Inspect the actual position coordinates
This direct calculation plots the original sinusoidal position encoding over integer token positions zero through 127. Even coordinate 2i uses sin(position / 10000^(2i/d)); its following odd coordinate uses cosine with the same denominator. The width d determines the frequency schedule. The plot joins discrete samples with straight lines; tokens do not occupy the intermediate positions drawn by those lines. This page computes position vectors alone. It does not add token embeddings, project queries and keys, or run attention.
Control width, selected coordinates and the probe
Model width accepts even integers from two to 256. Even width keeps every sine coordinate paired with a cosine coordinate. Dimensions accepts a comma-separated list of up to eight unique nonnegative coordinate indices strictly below that width. Duplicate indices appear once. Changing width preserves the typed list, so a formerly valid high index can produce an explicit error until corrected. Probe position selects an integer from zero to 127; the dashed line moves there and the table reports the value of each selected coordinate. Reset defaults restores width 64, dimensions 0,1,32,33 and probe zero. Results remain inspectable indefinitely; no playback clock or speed setting is needed for this direct mathematical instrument.
Reproduce two exact pairs
Reset defaults. At position zero, dimensions zero and 32 have value zero; dimensions one and 33 have value one. Set Probe position to one. The first pair becomes sin(1) and cos(1), approximately 0.841470984808 and 0.540302305868. Dimensions 32 and 33 use frequency 0.01 radians per token, so their values become approximately 0.009999833334 and 0.999950000417. Within either complete pair, the sum of squared coordinates equals one within floating-point error. Selecting dimensions 0,1 therefore compares phase, not different frequencies. Select 0,2 to compare frequencies instead.
Test what increasing width changes
Keep dimensions 0,1,32,33 and position one, then change width from 64 to 128. The first pair stays at frequency one; dimensions 32 and 33 now have frequency 0.1. The same coordinate number refers to a different fraction of the wider vector, so its oscillation becomes faster in this comparison. At width 64, the wavelength column shows about 6.283185307 token units for dimension zero and 628.3185307 for dimension 32. Wavelength describes the continuous sinusoid; a noninteger wavelength is not an exact repeat period in the integer sample sequence.
Read results and exercise failure boundaries
The table identifies each curve by coordinate, sine or cosine, frequency, wavelength and current numeric value. Open Full precision and all 128 samples for the unrounded computed evidence; its scrollable JSON includes every plotted integer sample. Table and evidence regions can receive keyboard focus and scroll horizontally on narrow phones. Enter width 63, dimension 64 at width 64, a trailing comma, or probe 128: the page reports the problem and clears the drawing and table instead of showing stale results. Reset defaults recovers a valid fixture. JavaScript double precision and rounded table text introduce small numerical differences. Distinct wavelengths alone do not prove better extrapolation, uniqueness at arbitrary distances, or quality of a trained language model.