Why architecture matters here
Without a gateway, MCP deployments hit three walls. The first is configuration sprawl: each client surface (IDE plugin, chat product, CI agent, internal assistant) carries its own list of servers, credentials, and versions. Adding a server means touching every client config; rotating a credential means finding every copy. The second is security incoherence: each MCP server implements its own auth, so the organization's effective access policy is the union of forty independent implementations, some of which are a stdio binary with filesystem access and no auth at all. The third is zero observability: tool calls flow point-to-point, so there is no single log answering who called which tool with what arguments — the first question in every incident involving an agent.
Quotas and rate limits share the same story: without a chokepoint, a runaway agent loop hammering an expensive backend is discovered by the invoice. The gateway is where per-principal and per-backend quotas become enforceable — and where a misbehaving agent can be throttled or cut off in one place, during the incident, rather than by hunting down which of a dozen client configs to revoke.
The gateway pattern matters because it converts those three problems into one component's responsibilities. Clients configure exactly one server. Policy is written once, against a merged catalog, and enforced on every call regardless of which backend serves it. Audit is a side effect of the data path. This is the same consolidation story as API gateways a decade earlier — with the twist that MCP is stateful (sessions, subscriptions, server-initiated notifications and sampling requests), so the gateway cannot be a stateless reverse proxy; it must actively manage session state on both faces.
There is also a velocity argument that lands with platform teams: a gateway makes backends swappable and composable. An internal team can replace a vendor's MCP server with their own, keep the namespaced tool names stable, and no client notices. New servers onboard by registering with the gateway — subject to policy review — rather than by lobbying every client team for a config change. The gateway becomes the marketplace surface for the organization's tools.
Finally, there is a quality argument that is easy to miss until an agent hits it: context economy. An agent connected to forty raw servers sees every tool of every server in its context window — thousands of tokens of descriptions, most irrelevant to the task, all of it prompt-injection surface. A gateway that filters the catalog per principal, per surface, or even per task (exposing curated toolsets rather than raw unions) directly improves agent accuracy and cost. The gateway is not just plumbing; it is the curation point where someone finally decides which tools an agent should see, separate from which tools exist.
The architecture: every piece explained
The client-facing server face speaks full MCP: it answers initialize with merged capabilities, serves tools/list, prompts/list, and resources/list from its aggregated catalog, and emits listChanged notifications when any backend's catalog moves. The merge is policy-filtered per principal: two different users initializing against the gateway can legitimately see different tool catalogs, because the policy engine prunes the merge before it is returned.
The namespace mapper prevents collisions and enables routing. Tools are exposed as github.create_issue, jira.create_issue — backend alias plus original name — and the mapper maintains the reverse mapping so an incoming tools/call routes to the right backend with the original name restored. Resource URIs and prompt names get the same treatment. The mapper must be stable across restarts (aliases live in the registry, not in memory) because clients cache tool names in conversation history; renaming a namespace mid-session breaks an agent's in-flight plan.
The auth broker is the security heart. The client authenticates to the gateway once (OAuth token, mTLS, API key). Per backend, the broker exchanges that identity for a scoped credential: an OAuth token exchange for SaaS backends, a service account with the caller's identity in a header for internal ones, nothing for local stdio servers the gateway itself spawns. The point is that backend credentials never live in client configs, and the blast radius of a leaked client token is the gateway's policy for that principal, not the union of all backend permissions.
The session mapper maintains the stateful correspondence: one client session fans out to lazy per-backend sessions, opened on first use and kept alive with the backend's transport semantics. Server-initiated traffic — progress notifications, resource-update events, sampling requests — flows backward through the map to the owning client session. The transport bridge normalizes the wire: stdio child processes, streamable HTTP with SSE, legacy HTTP+SSE backends — all presented to clients as one streamable HTTP endpoint. Around all of it sit health checks and circuit breakers per backend, and the audit plane that records every call, its principal, its arguments (redacted per policy), its backend, and its latency.
Two protocol features deserve special gateway handling. Sampling — a backend asking the client's model to complete a prompt — crosses the trust boundary in the dangerous direction: the gateway should police it with explicit per-backend policy (allowed, denied, or human-approved) rather than blind forwarding, because a compromised backend could otherwise use the client's model and context as an oracle. Elicitation — a backend requesting structured input from the user mid-call — must be relayed with clear attribution of which backend is asking, so users are not phished by one server impersonating another through the gateway's single trusted face.
End-to-end flow
Follow a session. An engineer's IDE agent connects to mcp.corp.example and runs initialize. The gateway authenticates the OAuth token, resolves the principal and their policy tier, and returns merged capabilities. The agent calls tools/list: the gateway serves it from its catalog cache — 214 tools across 31 backends, pruned by policy to the 90 this principal may see, each namespaced. Nothing has touched a backend yet; the catalog cache was built from earlier tools/list calls and is kept fresh by listChanged subscriptions to each backend.
The agent decides to call github.create_issue. The gateway's policy engine evaluates the call — principal, tool, argument patterns (a rule flags repo names outside the allowed org), rate quota — and passes. The namespace mapper resolves the backend and restores the tool name to create_issue. The session mapper finds no live session to the GitHub backend for this client and opens one: transport is streamable HTTP, so the gateway runs initialize against the backend, presenting a token the auth broker just minted by exchanging the engineer's gateway token for a GitHub-scoped credential. The call forwards; the backend streams a progress notification ('creating issue…') which the gateway relays backward, re-tagged with the client's request ID; the result returns and the audit plane logs the full round trip with arguments redacted per the logging policy.
Mid-session, a backend deploys a new version and its tool list changes. It emits listChanged; the gateway re-fetches, diffs the catalog, updates the registry, and forwards listChanged to every client whose policy-visible catalog changed. Clients re-list on their next turn and see the new tools. Later, the DB backend stops answering health probes; its circuit breaker opens. Calls to db.* tools now fail fast with a structured error ('backend unavailable, retry advised') instead of hanging an agent for its full timeout, and the catalog optionally marks the namespace degraded. When probes recover, the breaker half-opens, a canary tools/list succeeds, and traffic resumes — the client sessions never noticed beyond one fast failure.
The gateway's own deploys are part of the flow story, because every agent session in the company runs through it. A rolling upgrade must drain gracefully: stop accepting new sessions on the old instances, let in-flight tool calls complete within their timeout budgets, and either migrate long-lived sessions (session state in the shared store makes reconnection a resume, not a restart) or signal clients to re-initialize. MCP clients handle a clean close-and-reconnect well; what they handle badly is a half-dead gateway that accepts calls and never answers. Deploy tooling should therefore treat 'connections drained' as a gate, not a hope — the difference is visible in every IDE in the building.