Why it matters

Downtime affects users + business. HA is production requirement.

Advertisement

The architecture

Multiple instances behind LB.

Multiple regions for geo-redundancy.

Health checks route around unhealthy.

HA patternsMulti-instancein regionMulti-regionacross geoHealth + failoverroute around failureModel API failures require cross-region + cross-provider redundancy at some scale
HA components.
Advertisement

How it works end to end

Health checks: HTTP endpoint. LB routes only to healthy.

Multi-region: primary + standby; automatic failover on region outage.

Model API redundancy: fallback to alt provider on outage.

Session state HA: use multi-region DB (Spanner, DynamoDB Global).