Dictionary Learning for NN

Why it matters

Dictionary learning frames SAE. Understanding shapes theory.

The architecture

Learn overcomplete dictionary + sparse codes.

Applied to NN activations.

SAE is one instance.

How it works end to end

Sparse coding + KSVD classical.

SAE modern deep-learning instance.

Gated + Top-K + JumpReLU variants.