An API Gateway sits in front of your services and handles cross-cutting concerns: routing, authentication, rate limiting, request/response transformation, observability. Done well, it lets services focus on business logic. Done poorly, it becomes the bottleneck.
Core responsibilities
Routing: map URL path → backend service. Authentication: verify JWT/API key. Rate limiting: per-user, per-IP, per-endpoint. TLS termination: HTTPS in, HTTP out. Observability: log + metrics + trace ID injection.
Architecture pattern
Edge → Gateway tier (Envoy/NGINX/Kong fleet) → Service mesh / direct → backend services. Gateway is stateless; scale horizontally. State (rate-limit counters, JWT keys) in Redis or a CRDT store.
Performance budget
Latency overhead: <5ms p99 added by the gateway. CPU: ~1 vCPU per 10K rps on Envoy. Memory: <500MB per instance regardless of route count. If your gateway is the bottleneck, you've added too many features.
Don't do business logic here
Common mistake: aggregating multiple backend calls in the gateway (BFF pattern). Couples gateway to business. Better: a dedicated BFF service behind the gateway. Gateway stays thin and stable.
Tool choice
Envoy (CNCF, hottest for K8s, Istio uses it). Kong (NGINX-based, GUI for non-developers). AWS API Gateway (serverless, easy if all-in on AWS). Tyk, KrakenD (open-source alternatives). Pick by team familiarity; all do the basics well.