Why architecture matters here
The architecture matters because the alternative — a resilience-and-security library linked into every service — fails on consistency and on upgrade velocity, the two things that matter most at scale. A library must exist for every language the fleet uses, and those implementations drift: the Go client's retry semantics differ subtly from the Java one, and nobody notices until an incident. Worse, changing a policy means shipping new binaries everywhere, so a fleet-wide timeout change becomes a months-long migration. The mesh centralizes the behavior in one proxy implementation and one control plane, so a policy change is a config push, not a redeploy of a hundred services.
Security is the reason many organizations adopt a mesh at all. In a flat network, any service can call any other and the traffic between them is often plaintext — a single compromised pod can move laterally at will. A mesh makes mutual TLS the default: every workload gets a cryptographic identity, every connection is encrypted, and both ends authenticate each other. Authorization policy — 'only the checkout service may call the payments service' — is then enforced at the proxy on identity, not on an IP address that can be spoofed or reused. Encryption in transit and zero-trust authorization become properties of the infrastructure rather than features each team must remember to build.
Observability is the third driver, and it is nearly free once traffic flows through a uniform proxy. Because every request crosses a sidecar, the mesh can emit the same golden signals — request rate, error rate, and duration — for every service without any application instrumentation, and it can propagate trace context so a request can be followed across service boundaries. The result is a consistent, service-to-service view of the whole system that would be impossible to assemble if each team instrumented its calls differently. You get a map of who calls whom, how often, and how well, as a side effect of routing.
Finally, the mesh gives operators traffic control as a first-class capability. Canary releases, blue-green cutovers, fault injection for resilience testing, and circuit breaking are all expressed as declarative routing rules the control plane applies to the proxies — no application code changes to shift a percentage of traffic or to trip a breaker when a dependency degrades. That decoupling of deployment from release is what lets teams ship continuously and de-risk rollouts, and it is why the mesh, despite its real operational cost, has become a standard layer in large microservice platforms: it turns cross-cutting networking concerns into governed, uniform, centrally-programmable infrastructure rather than a thousand slightly-different library calls scattered across the fleet.
The architecture: every piece explained
The sidecar proxy is the data plane and the unit everything else configures. It is a high-performance L4/L7 proxy — Envoy is the de facto standard — deployed as a second container in every pod. Traffic is redirected into it transparently (via iptables rules or a CNI plugin) so the application needs no awareness of it. On the data path the proxy terminates and originates mTLS, applies timeouts and retries, load-balances across endpoints, trips circuit breakers, and records metrics and traces. Because it is identical everywhere, one behavior is enforced uniformly across the fleet.
The control plane is the brain that configures every proxy. It watches the platform's service registry (Kubernetes endpoints, say), reads the operator's declarative policy, and compiles both into per-proxy configuration: which endpoints exist for each service, what routing rules apply, which certificates to present. It then streams that configuration to the proxies and keeps it current as pods scale and policies change. Istio's istiod, Linkerd's controller, and Consul's servers all play this role.
xDS is the protocol that connects the two. It is a set of gRPC discovery services — LDS for listeners, RDS for routes, CDS for clusters, EDS for endpoints — over which the control plane pushes incremental configuration to each proxy and the proxy acknowledges it. xDS is what makes the mesh dynamic: when a new replica of a service appears, EDS pushes the new endpoint to every proxy that might call it, within seconds, with no restart. Understanding that config flows continuously over xDS is key to understanding both the mesh's power and its failure modes.
Workload identity is what makes security real. A certificate authority in the control plane issues each workload a short-lived X.509 certificate encoding a SPIFFE identity — a URI like spiffe://cluster/ns/checkout/sa/checkout — and rotates it automatically every few hours. mTLS handshakes use these certificates so both peers prove who they are cryptographically, and authorization policies are written against the SPIFFE identity rather than an IP. Ingress and egress gateways are dedicated proxies at the mesh boundary: the ingress gateway admits north-south traffic from outside into the mesh under the same policy and TLS regime, while the egress gateway funnels outbound calls through a controlled, observable choke point so external dependencies are governed rather than called ad hoc. The telemetry pipeline collects the RED metrics, access logs, and trace spans every proxy emits and ships them to Prometheus, a tracing backend, and dashboards. Together these pieces — sidecar, control plane, xDS, identity, gateways, and telemetry — are the full mesh, and each addresses a distinct cross-cutting concern that would otherwise live, inconsistently, in application code.
End-to-end flow
Trace a single call from the checkout service to the payments service. Checkout makes a plain HTTP request to payments.svc. The iptables rules in its pod redirect that connection to its local sidecar before a byte leaves the pod. The sidecar looks up its routing configuration — pushed earlier over RDS — and sees a rule for payments: a timeout of 800ms, a retry budget of two attempts on connect failures, and a destination cluster of healthy payments endpoints supplied by EDS.
The sidecar picks a payments endpoint by its load-balancing policy and opens a connection. Before any request data flows, the two sidecars perform an mTLS handshake: checkout's proxy presents its SPIFFE certificate, payments' proxy presents its own, each verifies the other against the mesh CA, and the connection is now encrypted and mutually authenticated. Payments' sidecar then consults its authorization policy — 'may spiffe://.../checkout call this method?' — and, because the policy allows it, forwards the request to the payments application on localhost.
The payments app processes the charge and replies. Its sidecar records the response code, the latency, and a trace span; checkout's sidecar does the same on its side and propagates the trace header so the two spans join into one distributed trace. Both proxies emit RED metrics tagged with source and destination identities. If payments had returned a retryable error or exceeded the timeout, checkout's sidecar — not the application — would have applied the retry or failed fast, entirely per the pushed policy.
Now the control-plane side of the story. Suppose an operator wants to canary payments v2. They apply a routing rule: 90% of payments traffic to the v1 subset, 10% to v2. The control plane compiles this into new RDS configuration and pushes it over xDS to every proxy that routes to payments. Within seconds, checkout's sidecar — with no redeploy of checkout — begins splitting traffic 90/10, and the telemetry pipeline shows v2's error and latency signals separately so the operator can watch the canary. If v2 misbehaves, the operator sets the split back to 100/0 and the change propagates just as fast. The application never learned that any of this happened: the release was a control-plane operation, the security was a data-plane property, and the observability came for free — which is exactly the separation of concerns the mesh exists to provide.