Why it matters

SSMs offer linear scaling + parallel training. Foundation of Mamba, S4.

Advertisement

The architecture

Linear recurrence.

Discrete or continuous.

Parallel convolution training.

State space modelsLinear recurrenceh_t = A h_{t-1} + B x_tConvolutionfor trainingRecurrencefor inferenceS4 (Gu 2021); Mamba (2023); linear scaling; parallel training
State space models.
Advertisement

How it works end to end

S4 (Gu 2021).

Mamba (Gu-Dao 2023).

Parallel convolution.