Dropless MoE

Why it matters

Dropless MoE preserves data. Understanding shapes modern MoE.

The architecture

Standard MoE drops overflow.

Dropless: use variable-batch expert kernels.

Sparse GEMM kernels.

How it works end to end

MegaBlocks: sparse matmul.

Grouped GEMM for variable batches.

No capacity waste.