Why it matters

SSMs are alternative to attention. Foundation of Mamba family.

Advertisement

The architecture

Continuous formulation.

Discretize: ZOH or bilinear.

Compute via convolution or recurrence.

SSM structureContinuous SSMODE formDiscretizeZOHComputeconv or recurConvolutional form for training; recurrent form for inference
SSM.
Advertisement

How it works end to end

Convolutional form: fast training.

Recurrent form: constant memory inference.

HiPPO: initialization for long dependencies.