Why it matters

TCO drives buy vs rent decisions. Understanding shapes financial planning.

Advertisement

The architecture

Capital: H100 ~$30-40K. B200 ~$60K+. Server ~$300K for 8-GPU node.

Power: 700W per H100 × 24/7 = ~$800/year @ $0.13/kWh + cooling PUE.

GPU TCO componentsCapital + financeGPUs + servers + netPower + cooling700W × 8760 hrsOps + softwareeng + software3-year TCO: power often 30-50% of capital
TCO breakdown.
Advertisement

How it works end to end

Cooling: 1.3-1.5 PUE typical.

Networking: IB switches expensive.

Ops: skilled engineers to run large clusters.

Buy vs rent: sustained load → buy; spiky/variable → rent.