Why it matters
Sharing GPUs increases utilization dramatically. In cloud infrastructure, a single H100 can serve many inference customers with MIG. On training clusters, MPS enables concurrent debugging and development on shared GPUs.
The architecture
Time-slicing: default behavior. Multiple CUDA contexts share the GPU by taking turns. Simple but incurs context switch overhead per switch.
MPS: a server-client model where a single CUDA context serves multiple client processes. Kernels from different clients run concurrently on the same GPU. Great for many small workloads.
How it works end to end
MIG: physical partitioning available on A100/H100/B100. A single GPU can be partitioned into up to 7 GPU instances, each with its own memory and compute resources. Instances are hardware-isolated — failures don't cascade.
MIG use cases: cloud multi-tenant inference (each tenant gets a slice); Kubernetes GPU scheduling with fine granularity; dev environments where devs need isolated GPUs.
Sharing trade-offs: time-slicing gives no isolation and overhead; MPS gives concurrent but no isolation; MIG gives isolation but reduced total throughput.