Core concept
Topology-aware. Uses NVLink, NVSwitch, InfiniBand, PCIe. Auto-selects fastest path.
Advertisement
How it works
Debugging distributed issues can be painful. Tuning tricky.
Advertisement
Trade-offs + gotchas
All distributed training. FSDP + DDP use NCCL under hood.