Why architecture matters here
The gateway is architectural because the concerns it owns are genuinely cross-cutting — they apply to every request regardless of which service ultimately handles it — and cross-cutting concerns implemented per-service drift, diverge, and rot. If ten services each verify auth tokens, you have ten subtly different implementations, ten places a security fix must land, and ten chances for one of them to get it wrong. Centralizing auth at the edge makes it one implementation, one place to patch, one consistent policy. The same argument holds for rate limiting (a global view of a client's usage across services is only possible at a shared choke point), for TLS termination (one place to manage certificates), and for observability (one place to assign a request ID and start a trace that follows the call through every backend).
The second reason the gateway is a distinct architectural layer is that it decouples the client's contract from the backend's topology. Clients call stable, versioned paths at one hostname; behind the gateway, services can be split, merged, moved, or reimplemented in another language, and as long as the gateway's routing table is updated the clients never notice. This is what lets a team refactor a monolith into microservices incrementally: the gateway presents the same external API while the routing quietly shifts paths from the old service to the new ones. The gateway is the seam that makes internal reorganization invisible to the outside.
But centralization concentrates both failure and latency, and that is the constraint every design decision answers to. Because all traffic passes through it, the gateway must be at least as available as the most available service behind it — usually more so, since its outage is total — which forces horizontal scaling, health-checked redundancy, and careful blast-radius control. Because it is on every request's critical path, its own processing must be cheap and bounded: a gateway that does an expensive synchronous lookup on every call has added that cost to the entire system. The rule that follows is to keep the gateway thin — fast, stateless where possible, delegating anything heavy — so that the price of a shared front door stays small.
There is a subtler architectural boundary the gateway must hold, and violating it is the most common way these systems decay: the line between policy and business logic. Policy — is this caller authenticated, are they within quota, does this path route here, should this field be stripped from the response — is generic, request-shaped, and legitimately cross-cutting, and it belongs at the edge. Business logic — how a price is computed, whether an order is valid, what a recommendation should be — is specific to a domain and belongs in the service that owns that domain. The two feel similar when you are staring at a request because both manipulate the request, but they change for entirely different reasons and at entirely different rates: policy changes when security or contracts change, business logic changes when the product changes. A gateway that starts making product decisions becomes a place every team must edit to ship a feature, a deployment bottleneck that couples unrelated services through a shared hot path, and eventually a distributed monolith wearing a gateway's clothes. Keeping the gateway to policy is not fussiness; it is what preserves the independent deployability that motivated splitting into services at all.
The architecture: every piece explained
Top row: the request pipeline, executed in order on every call. Clients — browsers, mobile apps, partner servers — connect to a single hostname. The gateway terminates TLS and then authenticates: it verifies the credential (a JWT signature, an API key lookup, an OAuth token introspection) and rejects anything unauthenticated before it touches a backend. Next it enforces rate limits — per API key, per tenant, per route — using a shared counter so a client's global quota is honored no matter which service they hit. Only then does the router map the request path and method to a backend service and forward it. Doing auth and limits before routing means abusive or unauthenticated traffic is rejected at the cheapest possible point, never reaching the services.
Middle row: the optional shaping stages. Transform rewrites requests and responses — adding headers, stripping internal fields, translating between an external API shape and an internal one — so backends can speak their natural protocol while clients see a stable contract. Aggregation lets one client call fan out to several backends and compose their results into a single response, the backend-for-frontend pattern that spares a mobile app from making six round trips over a slow network. The circuit breaker protects the whole system from a failing backend: when a service starts timing out or erroring past a threshold, the gateway stops sending it traffic and fails fast (or serves a fallback), so one sick service cannot exhaust the gateway's connections and stall requests bound for healthy services. Behind these sit the backends — the microservices, now free of all this cross-cutting machinery.
Bottom rows: the supporting planes. Observability is not optional at the gateway: because every request passes through, it is the natural place to assign a correlation ID, emit structured access logs, record latency and status metrics per route, and start a distributed trace that threads through every downstream hop. The control plane is deliberately separate from the data plane: it is where routes, rate-limit policies, auth config, and API keys are managed and pushed to the gateway instances, and it must be able to fail without taking the data plane with it — a gateway that cannot serve traffic when its config store is briefly unreachable has coupled the two planes fatally. The ops strip captures the discipline: hold a tight latency budget, keep the gateway thin, version routes so clients migrate gracefully, and protect the control plane so config changes are safe and its outages are survivable.
End-to-end flow
Follow a mobile app loading a product screen. The app sends one request — GET /screen/product/123 — to the gateway over TLS. The gateway terminates TLS, reads the bearer token, verifies its signature and expiry locally without a network round trip, and extracts the user and tenant. It checks the rate limiter: this tenant is well under quota, so the request proceeds. The router recognizes the screen path as an aggregation route rather than a simple proxy. Everything so far took well under a millisecond of gateway work, and no backend has been touched yet.
Now the aggregation stage fans out. The single client call becomes three parallel backend calls: the catalog service for product details, the pricing service for the tenant-specific price, and the inventory service for stock. The gateway issues them concurrently, waits for all three (or applies a per-call timeout and degrades gracefully if one is slow), and composes their responses into the single JSON the screen needs. On the way out the transform stage strips internal fields — the catalog's supplier cost, the inventory service's warehouse IDs — that the client must never see. The app made one round trip over a slow mobile network instead of three, and the composition and field-stripping happened at the edge where it is cheap.
Suppose the inventory service is having a bad day. Its latency has climbed and it is returning errors. The circuit breaker for that backend has tripped after too many failures, so instead of piling more requests onto a drowning service and waiting out its timeouts, the gateway fails that leg fast and returns the screen with the product and price present and stock marked temporarily unavailable. The user still sees a useful page; the inventory service gets breathing room to recover rather than a thundering herd; and — critically — the catalog and pricing requests for other users are unaffected because the gateway's connection pool was not exhausted holding open doomed inventory calls. One sick backend degraded one field, not the whole system.
Now the control-plane scenario that separates a robust gateway from a fragile one. The platform team ships a new pricing service and needs traffic to shift to it. In the control plane they update the route for /pricing to point at the new service, roll it out to a fraction of gateway instances, watch the metrics and traces the gateway emits, and — seeing healthy latency and error rates — complete the rollout. Clients noticed nothing; the external contract never changed. Then imagine the config store the control plane uses goes briefly offline. A well-built gateway keeps serving traffic from its last-known-good config cached in memory: the data plane does not depend on the control plane being reachable moment to moment. Only new config changes are blocked until the store returns, which is exactly the right failure mode — you cannot reconfigure, but you never stop serving. A gateway that instead needed a live control-plane lookup per request would have turned a config-store blip into a full outage, which is why the two planes are kept separate and the data plane is built to survive the control plane's absence.