Why it matters
Bottleneck theory frames generalization. Understanding shapes intuition.
Advertisement
The architecture
Fit phase: memorize + label.
Compression phase: discard nuisance.
Generalization from compression.
Advertisement
How it works end to end
Mutual information (X; T) vs (T; Y).
Two-phase training dynamics.
Explains generalization heuristically.