Interactive labs
All 4 labs in this category
Advertisement
LAB · 01
Feed-Forward Layer (MLP)
Hidden dimension expansion + activation + projection back.
Open lab →LAB · 02
Mixture of Experts Routing
Router picks K experts per token. See activation patterns.
Open lab →LAB · 03
Multi-Head Attention
See how heads specialize on different patterns.
Open lab →LAB · 04
Positional Encoding — Sinusoidal vs RoPE
Two ways to inject position information.
Open lab →