Why it matters
RetNet unifies training/inference forms. Understanding shapes design.
Advertisement
The architecture
Retention: attention-like with exponential decay.
Three equivalent forms.
Advertisement
How it works end to end
Parallel: matrix ops for training.
Recurrent: constant memory inference.
Chunkwise: sublinear memory for long sequences.