Why it matters

Capacity estimation shapes provisioning. Understanding shapes cost.

Advertisement

The architecture

Bench prefill + decode + KV.

Compute concurrent capacity.

Apply SLO discount.

Capacity estimationBench per-GPUthroughput + latencyConcurrent estat SLOProvisioningGPUs neededReal workloads have long tail; add 30-50% safety margin over benchmarks
Capacity.
Advertisement

How it works end to end

Benchmark: run realistic workload.

Compute max concurrent users.

Apply p99 discount.

Safety margin 30-50%.