Core concept
Not bandwidth-optimal for large. Total transfer higher than ring.
Advertisement
How it works
NCCL picks tree vs ring based on message size + topology.
Advertisement
Trade-offs + gotchas
Latency-sensitive small collectives. Many workers.