Feature Disentanglement

Why it matters

Disentanglement shapes interpretable features. Understanding shapes research.

The architecture

Neurons often polysemantic (multiple concepts).

Overcomplete dictionary yields cleaner features.

SAE decomposes.

How it works end to end

Elhage et al. 2022 superposition.

Bricken et al. 2023 SAE.

Gated SAE reduces reconstruction error.