⟨a, b⟩ = |a|·|b|·cos(θ). Drag vector tips to explore.

What you're seeing

In 2D: drag the tip of either vector. The plot shows the dot product, both magnitudes, and the angle. Normalizing → vectors of length 1 → dot product = cosine similarity.

Attention's core: similar query/key vectors → large dot product → high attention weight.

★ KEY TAKEAWAY
Dot product = |a|·|b|·cos(θ). Aligned vectors → big positive. Perpendicular → zero. Opposite → negative.
▶ WHAT TO TRY
  • Drag both vector tips and watch ⟨a,b⟩ change.
  • Make them perpendicular — dot product hits 0.
  • Toggle Normalize to see cosine similarity (length-invariant).
← Back to Transformer Math & CPU SLM