Why architecture matters here
PCIe surprises are usually topology surprises. A GPU pair on different root complexes can't P2P, so transfers bounce through host memory at half bandwidth. NUMA misalignment means the CPU handling GPU 0 is across the QPI link.
The architecture matters because tuning happens at server layout, driver settings, and application-level topology awareness.
With the pieces in mind, you get expected bandwidth instead of paper specs.
The architecture: every piece explained
The top strip is the physical stack. Host CPU issues memcpy operations. System memory is DDR arranged in NUMA regions. PCIe root complex is the switching fabric with lanes per socket. GPU memory is HBM3 on-package.
The middle row is the mechanics. Lanes + width: each PCIe slot has a width (x8, x16); gen5 doubles gen4 bandwidth per lane. GPU-GPU P2P works when GPUs sit on the same root complex and P2P is enabled. Host bounce happens when P2P is unavailable — CPU serves as intermediary. GPUDirect Storage transfers NVMe → GPU without host bounce.
The lower rows are ops. NUMA locality ensures the CPU thread controlling a GPU is on the same socket as the PCIe root the GPU attaches to. Metrics track PCIe bandwidth and errors. Ops validates topology and manages firmware + drivers.
End-to-end flow
End-to-end: a training script needs to send activations between GPU 0 and GPU 1. Same root complex, P2P enabled → direct transfer at 50 GB/s on PCIe gen5 x16. GPU 0 and GPU 2 on different sockets → no P2P; host bounce at 25 GB/s. NVMe → GPU dataset load uses GPUDirect Storage at 12 GB/s, no host CPU involvement. NUMA binding: CPU threads for GPU 0 pinned to socket 0. Metrics validate expected bandwidth; nothing under 70% of paper spec.