Why architecture matters here

Generation choices matter because software written for Hopper misses Blackwell wins (FP4). Architecture matters because SM + precisions + interconnect all shift.

Advertisement

The architecture: every piece explained

The top strip is generations. Hopper H100. H200. Blackwell. GB200.

The middle row is changes. SM redesign. FP formats. Memory bandwidth. NVLink.

The lower rows are ops. Training vs inference. Power + thermals. Ops — capacity + tuning + supply.

Ada Lovelace / Blackwell — SM changes + FP8 + memory + generational tradeoffsthe arc from Hopper to BlackwellHopper H100132 SM + FP8 + TMAHopper H200141 GB HBM3eBlackwell B100/B200dual-die + FP4GB200 grace-blackwellCPU + GPU + NVLinkSM redesignclusters + asyncFP formatsFP16/BF16/FP8/FP4Memory bandwidthTB/s per GPUNVLink generations5.0 chip-to-chipTraining vs inferencetuned for bothPower + thermalskW-classOps — capacity planning + tuning + supplyclustershrinkboostconnectspecializecoolcooloperateoperate
Recent NVIDIA generations: Hopper → Blackwell.
Advertisement

End-to-end flow

End-to-end: LLM inference workload targets Blackwell B200. FP4 kernels used for weight matmul; FP8 for activations. HBM3e bandwidth utilization 80%. NVLink 5.0 fabric for tensor parallelism. Power planned per rack.