Why it matters

Data movement between CPU and GPU is often the hidden bottleneck. Understanding PCIe shapes efficient GPU code.

Advertisement

The architecture

PCIe Gen4 x16: ~32 GB/s each direction.

PCIe Gen5 x16: ~64 GB/s each direction.

PCIe fundamentalsPCIe lanesx16 typical for GPUBandwidth32-64 GB/s each dirBottleneckCPU-GPU transferHBM bandwidth (2-3 TB/s) >> PCIe; minimize host-device transfers
PCIe basics.
Advertisement

How it works end to end

Server topology: 8 GPUs share limited PCIe lanes to CPU. Communication between GPUs often via NVLink instead.

DMA: GPUs use DMA to move data over PCIe without CPU involvement.

P2P over PCIe: some CPUs support GPU-to-GPU DMA over PCIe.