Advertisement
⟨a, b⟩ = |a|·|b|·cos(θ). Drag vector tips to explore.
What you're seeing
In 2D: drag the tip of either vector. The plot shows the dot product, both magnitudes, and the angle. Normalizing → vectors of length 1 → dot product = cosine similarity.
Attention's core: similar query/key vectors → large dot product → high attention weight.
★ KEY TAKEAWAY
Dot product = |a|·|b|·cos(θ). Aligned vectors → big positive. Perpendicular → zero. Opposite → negative.
▶ WHAT TO TRY
- Drag both vector tips and watch ⟨a,b⟩ change.
- Make them perpendicular — dot product hits 0.
- Toggle Normalize to see cosine similarity (length-invariant).