Interactive labs
All 5 labs in this category
Advertisement
LAB · 01
Feed-Forward Layer (MLP)
Hidden dimension expansion + activation + projection back.
Open lab →LAB · 02
LayerNorm vs RMSNorm
Both stabilize activations; RMSNorm skips mean centering.
Open lab →LAB · 03
Mixture of Experts Routing
Router picks K experts per token. See activation patterns.
Open lab →LAB · 04
Multi-Head Attention
See how heads specialize on different patterns.
Open lab →LAB · 05
Positional Encoding — Sinusoidal vs RoPE
Two ways to inject position information.
Open lab →