Active-Active LLM Serving

Why it matters

Active-active shapes availability. Understanding shapes design.

The architecture

All regions serve.

LB distributes.

Failover: LB drains failed.

State replication if needed.

How it works end to end

Global LB + health checks.

Session affinity considerations.

State: stateless preferred.