Why it matters

Offload shapes large-model training. Understanding shapes ZeRO Infinity.

Advertisement

The architecture

Move state to CPU / NVMe.

Fetch during compute.

Prefetch to hide latency.

PCIe bandwidth constraint.

Offload math flowMove stateGPU -> CPU / NVMeFetch on demandwith prefetchPCIe bandwidthcritical constraintDeepSpeed ZeRO Offload + Infinity; FSDP CPU offload; huge model training
Offload math.
Advertisement

How it works end to end

DeepSpeed ZeRO Offload + Infinity.

FSDP CPU offload.

PCIe 4.0/5.0 bandwidth.

Overlap fetch + compute.