Core concept
H100 delivers ~1 PFLOP/s BF16 sparse. 10^25 FLOPs = 10^10 GPU-seconds = ~3000 GPU-years or 100k GPU × 100 days.
Advertisement
How it works
Chinchilla optimal ratio: ~20 tokens per parameter. Modern models over-train (100+ tokens/param).
Advertisement
Trade-offs + gotchas
Foundation model planning. Capacity + timeline estimation.