Multi-Head Split + Concat

Reshape projected values without losing features

This experiment begins with an already projected query matrix Q. It isolates the index rearrangement used to expose attention heads. It does not compute the projection from token embeddings, attention scores, softmax, values, or the final output projection. Token rows selects one to four rows; Projected width selects two to sixteen coordinates per row; Head count selects one to eight groups. The width must be divisible by the number of heads. A nondivisible width is an error, never permission to discard the last coordinates. Each head has width divided by head count coordinates.

Follow one value through every shape

Reset defaults creates two rows of width eight and two heads. Generated entries run consecutively from one to sixteen in row order. Splitting a row into contiguous groups gives head zero coordinates zero through three and head one coordinates four through seven. Q[1,6] equals fifteen. Its destination is Heads[1,1,2], which also equals fifteen: head one, token row one, local coordinate two. The shape sequence first groups columns as [rows,heads,headWidth], then reorders axes to [heads,rows,headWidth]. Calling both operations a simple reshape can conceal this axis permutation; the displayed index mapping makes it explicit.

Compare head counts with an exact invariant

Keep the generated two by eight matrix and change Head count to four. Q[1,6] now appears at Heads[3,1,0]. The number fifteen is preserved even though its head and local coordinate change. Select one head: its matrix equals the original Q. Select eight heads: each head has one feature per token. The Reconstructed column comes from concatenating the heads along the feature axis and must always equal the original value. Neither the number of elements nor their values changes. Extra arrays in this teaching implementation are not evidence of the allocation, contiguity or copy behavior of a tensor library.

Edit a concrete matrix and test rejected shapes

The Explicit Q matrix editor accepts a JSON array whose row and column counts match the controls. Each component must be a finite number between minus one thousand and one thousand. Set Token rows to one, width to four, heads to two and enter [[-2,0,3,7]]. The heads contain [-2,0] and [3,7]; reconstruction returns the original row. Clear the editor to return to generated consecutive values. Changing the dimensions preserves any typed matrix, so a shape mismatch clears the table and disables inspection until corrected. Width eight with three heads is also rejected. Reset defaults clears the editor and restores every parameter and normal inspection speed.

Control the inspection clock

Play highlights successive split mappings followed by concatenation mappings over sixty seconds at normal speed. Pause freezes the selected mapping. Restart pauses at the first mapping while preserving your matrix and dimensions. Next mapping advances one discrete mapping and pauses. Inspection speed offers quarter, half, normal and double speed; it changes viewing duration only. Inspection position seeks through the sixty-second trace and pauses. Completed tensors and all mappings are available immediately, including while paused, in the table and the expandable JSON evidence. Invalid experiments cannot play. Nothing autoplays, including for reduced-motion users.

Understand the architectural limit

This reversible rearrangement has no learned parameters and cannot make heads attend differently on its own. Learned projections and subsequent attention computations provide that behavior in a complete architecture. The lab uses equal-width query heads without a batch axis; grouped-query attention can use different query and key/value head counts and is outside its scope. Highlight color is accompanied by indices and a numeric value in the status text. Phone users can focus and scroll the mapping table and complete evidence without relying on color or a large canvas.