FP4 Training on Blackwell

Why it matters

FP4 is emerging cutting edge. Understanding shapes near-term training.

The architecture

FP4: 1 sign, ~3 bits split.

Micro-scaling: per-block scale factors.

Requires H100 successor (Blackwell).

How it works end to end

MXFP4: microscaling FP4 format.

Per-block scale factors.

Weights + activations in FP4.

Compute in FP4 tensor cores.