Why it matters

GPUDirect skips CPU for data transfer. Big win for training + inference.

Advertisement

The architecture

GPUDirect RDMA: NIC ↔ GPU.

GPUDirect Storage: NVMe ↔ GPU.

Bypass CPU.

GPUDirect deep diveGPUDirect RDMANIC ↔ GPUGPUDirect StorageNVMe ↔ GPUBypassCPU + sysmemNVIDIA GPUDirect Storage 1.0 (2021); MagnumIO for LLM data pipelines
GPUDirect deep dive.
Advertisement

How it works end to end

NVIDIA GDS 1.0.

NCCL uses.

MagnumIO stack.