A GPU cluster is bought in FLOPS and delivered in megawatts. Long before a single kernel launches, someone has to answer a chain of electrical questions: how many kilowatts does one rack pull, how many racks fit on a feeder, what happens when ten thousand accelerators start and stop a training step in lockstep, and — the question that actually decides when the cluster exists — when the utility will energize the service. This is the facility layer beneath every CUDA optimization: the transformers, busways, PDUs, and redundancy topology that turn a grid connection into powered GPUs, and the one software lever, power capping, that lets the compute side push back.
From kilowatt racks to megawatt halls
Datacenter electrical design spent two decades assuming a rack drew single-digit to low-double-digit kilowatts. A hall built around 5–10 kW cabinets spreads its load over a large floor, and floor space, not amperage, is the scarce thing. Dense accelerator racks invert that completely: a fully populated GPU rack draws tens of kilowatts, and the densest current liquid-cooled designs are specified around 100 kW and beyond. The same power that used to occupy a row now occupies a single cabinet.
That inversion changes what a datacenter is. Capacity is sold and planned in kilowatts, not square feet; a hall is “full” when its electrical distribution is full, often with empty tiles remaining. It also forces the thermal decision — past roughly 30 kW per rack air becomes impractical and the design moves to rear-door heat exchangers, direct-to-chip liquid, or immersion — but that is a separate story; everything below stays on the electrical side.
Nameplate, derate, diversity — the sizing math
The arithmetic starts with the wrong number. A GPU server’s nameplate rating is the sum of what its power supplies can deliver, not what it draws; steady-state draw under a saturating training job is lower, and idle draw far lower. Sizing every feeder to nameplate strands capacity nothing will use. Sizing to measured average invites a trip the first time a job runs the fleet at full tilt.
Practice splits the difference: size protective devices and conductors from the nameplate and continuous-load rules (in the United States the NEC requires a continuous load to sit at no more than 80% of the branch-circuit rating), then apply a diversity factor when rolling racks up to a hall total, on the assumption that not every rack peaks simultaneously. The uncomfortable truth about AI training clusters is that this assumption is exactly the one that fails, which is the subject of the next two sections.
Synchronized training looks like a square wave
A conventional mixed workload is statistically smooth. Thousands of unrelated VMs peak and idle at uncorrelated moments, so hall draw is a gently undulating line and diversity factors hold. A single large training job is the opposite: every GPU in the cluster runs the same step, in the same order, synchronized by a collective at the step boundary.
The result is a load profile with sharp edges. During forward and backward compute the accelerators sit near their power limit. During the all-reduce, checkpoint write, or data-loader stall that follows, they drop toward idle together. Then they all resume together. From upstream the hall behaves less like a population of independent servers and more like one enormous device being switched on and off at the step frequency — a square wave whose amplitude is a large fraction of the whole facility’s load. Job launches, job completions, and cluster-wide failures make the edges steeper still.
What a step load breaks upstream
Electrical infrastructure tolerates magnitude far better than it tolerates rate of change. Several things degrade when a large fraction of a hall steps at once. Protective devices are coordinated on time–current curves; a steep collective inrush can push an upstream breaker toward its instantaneous region and cause a nuisance trip that no individual server caused.
Voltage and frequency wobble too. A large step draws hard on the source, and on-site generation feels it most: engine-generator sets are rated for how much load they can accept in a single step before frequency dips beyond tolerance, so a facility that rides through on gensets can be stable at full load yet unstable during ramps. Upstream of the meter, repeated large steps show up on the utility’s network as flicker and regulation duty, which is why interconnect agreements for AI campuses increasingly include ramp-rate and power-quality terms, not just a peak demand figure.
Medium voltage to the row
Delivering tens of megawatts at rack voltage is not physically sensible: current scales inversely with voltage and conductor cost scales with current, so power travels as far as possible at high voltage and steps down as late as possible. A campus takes utility service at transmission or medium voltage, distributes at medium voltage (commonly somewhere in the tens of kilovolts) across the site, and transforms down close to the load.
Higher rack density has pushed that transformation physically closer to the racks. Where a traditional design put large transformers in an electrical room feeding long low-voltage runs, dense halls favour distributed medium-voltage skids or transformers at the end of the row, keeping low-voltage conductor lengths short. It is the same optimization as a memory hierarchy: move the expensive high-current segment as close to the consumer as possible and keep the long haul cheap.
Busway or whips — sizing the feeder
The last low-voltage leg reaches racks in one of two ways. Traditional designs run individual circuits — “whips” — from a floor PDU or panelboard to each rack. Dense designs increasingly use overhead busway: a continuous enclosed bus running the length of a row, with tap-off boxes clamped on wherever a rack needs feeding.
The reason is not aesthetics. At high rack kW the whip approach needs many large conductors converging on a panel, and every capacity change is an electrician re-pulling cable. A busway is sized once for the whole row’s ampacity, and adding, moving, or upsizing a rack becomes a tap-off change. The design constraint moves up a level: size the busway from intended rack kW times racks per row, plus headroom for the next hardware generation, because the bus is far harder to replace than anything hanging off it. Undersize the row and the row is capped for its lifetime.
Redundancy topology: N, N+1, and 2N
Redundancy is described by how much spare capacity exists and how independently it is fed. N is exactly enough capacity to carry the load with nothing spare. N+1 adds one redundant unit to a set, so any single UPS module, transformer, or generator can fail or be serviced without dropping load. 2N duplicates the entire path: two independent power trains, each able to carry the full load, with dual-corded equipment fed from both.
The practical vocabulary that matters is concurrently maintainable (any component can be taken out for planned work without interrupting the load) versus fault tolerant (the facility survives an unplanned failure of any single component while still carrying load). AI halls complicate the economics: 2N is expensive per delivered watt, and a training job that checkpoints frequently may tolerate a rare interruption far better than an inference fleet under an availability SLO. Redundancy is a per-hall decision.
Inside the rack: PDUs and phase balance
Below the busway tap, a rack PDU distributes to outlets. Three details dominate. First, phase balance: rack feeds are typically three-phase, and servers hang off individual phases. If loading drifts so that one phase carries much more current than the others, that phase hits its limit while the feed as a whole looks half empty, and the imbalance propagates upstream as neutral current and transformer heating.
Second, dual corded equipment on an A/B pair must be loaded so either side can carry everything alone — in practice each side runs well under half its rating, or the failover that redundancy exists for trips the surviving feed. Third, metering: intelligent PDUs report per-outlet and per-phase current, and that telemetry is the difference between managing a hall to measured draw and managing it to nameplate guesses. Without branch-circuit visibility, operators strand capacity out of caution.
Power capping as a control lever
Power capping is where the compute layer gets a say in the electrical one. Datacenter GPUs expose a settable board power limit — through NVML, DCGM, or nvidia-smi -pl — and the on-board controller enforces it by throttling clocks. Capping is not free: because power scales superlinearly with clock and voltage, the first slice of power removed costs disproportionately little throughput, while deep caps hurt badly. That curve is the whole reason the lever is useful.
Three ways it gets used. As static headroom: cap every GPU modestly so a rack’s worst case fits its circuit, buying real rack density at modest performance cost. As a dynamic guardrail: a controller watching branch-circuit or hall telemetry lowers caps when draw approaches a limit, trading throughput for staying energized. And as ramp smoothing: staggering job launches and easing caps upward instead of releasing thousands of GPUs to full power in one instant, which blunts precisely the step edge that upstream equipment dislikes.
The utility interconnect gates everything
Every constraint above is solvable with money and engineering on a known schedule. The interconnect is not. Connecting a large new load requires the utility to study the impact, potentially reinforce transmission, procure long-lead equipment such as large power transformers and switchgear, and energize — a process routinely measured in years, with queues in some markets longer than the useful life of the GPUs being installed.
This is why capacity planning for AI infrastructure now looks like an electrical siting exercise. Operators chase locations with existing service, retired generation whose interconnection rights can be reused, or behind-the-meter generation; they phase deployments to match staged energization; and they negotiate curtailable terms, since a training job that can be paused is an easier customer to connect than a firm load. The scarce resource in AI compute is increasingly not silicon but an energized megawatt.