Why architecture matters here
The reason to care is that overload is not a linear problem — it has a cliff. Below capacity, adding load adds throughput; the agent keeps up, latency is flat. Above capacity, adding load does the opposite: throughput falls as latency explodes, because the agent now spends resources on context-switching, queue management, and work that will time out before it finishes. Worse, the useful output (goodput) can approach zero even while the agent is maxed out, because it is completing requests that the caller already gave up on. An agent without load shedding does not gracefully slow down at the cliff; it goes over it, and the fall is a collapse, not a slowdown.
The second reason is that in an A2A mesh, an overloaded agent is a source of contagion. When agent B slows under load, agent A — which is waiting on B — also slows, holding its own resources longer, so A's callers slow too. Meanwhile A's requests to B time out and A retries, adding more load to the already-overloaded B. This positive-feedback loop is how a localized hotspot becomes a mesh-wide outage, and it is why the failure is often metastable: even after the original traffic spike passes, the accumulated retries keep B pinned, and the system does not self-recover. Shedding early breaks the loop at its source — B rejects excess work cheaply instead of slowly, so A learns immediately to back off instead of piling on.
The third reason is that not all work is equally valuable, and overload is exactly when that difference matters most. A user-facing task that a person is waiting on is worth more than a background reconciliation job; a first attempt is worth more than a speculative retry; a task from a high-tier tenant may be contractually prioritized. When an agent must drop something, dropping the least important thing preserves the most value. This is why load shedding is inseparable from prioritization: shedding blindly (dropping whatever arrives when the queue is full) can drop the one request that mattered while admitting ten that did not. The architecture has to classify before it sheds.
Understanding this reframes capacity planning around a number most systems ignore: goodput under overload. The right question is not 'how much load can the agent take before it slows down' but 'when demand exceeds capacity, does the agent keep completing its most important work, or does it thrash?' A well-designed shedder makes the answer the former — as load rises past capacity, accepted throughput plateaus at the agent's real ceiling and excess is shed, so the completion rate of admitted work stays high and predictable no matter how far demand overshoots. That plateau, rather than a collapse, is the property that makes an agent safe to depend on.
There is also a caller-side reason that is easy to miss: a fast, honest rejection is more useful to the caller than a slow success. A caller that receives a 429 with a Retry-After header in five milliseconds can immediately try an alternative agent, degrade its own response, or schedule the work for later — it retains agency. A caller stuck on a request that will eventually time out after thirty seconds has none: it has already committed the latency budget and gets nothing back to act on. Shedding is therefore not just self-protection; it is a cooperative signal that lets the whole mesh route around a hot spot, which is only possible because the rejection is early and carries actionable information.
The architecture: every piece explained
Top row: the intake path where the decision is made. Caller agents send task requests to the receiving agent. Those requests hit admission control at the edge — the component that decides, before any real work is committed, whether to accept or reject. Admission control consults the priority classifier, which tags each request with a tier (user-facing versus background, high-tier tenant versus best-effort, first attempt versus retry). Accepted requests enter a bounded queue — and 'bounded' is the essential word: the queue has a finite maximum size, so it can fill, and its fullness is the primary signal that the agent is near capacity. An unbounded queue is the classic anti-pattern that turns overload into an out-of-memory crash instead of a clean rejection.
Middle row: how the shed decision is computed and communicated. The load signal is what admission control watches to know it is overloaded — queue depth is the most direct, but latency (are accepted tasks completing within their deadline?) and resource saturation (CPU, in-flight count) are common inputs too. When the signal crosses a threshold, the shed decision engages: reject incoming requests, lowest priority first, so background and speculative work is dropped while user-facing work is still admitted. A shed request gets a 429 with Retry-After — the backpressure contract that tells the caller both 'I refused this' and 'wait this long before trying again,' shaping the caller into cooperative backoff rather than an immediate hammering retry. Behind all this sits the worker pool, the protected capacity that the shedder exists to keep healthy.
Bottom rows: the two fates of a request and the ops surface. On the accepted path, a request that passed admission runs to completion with resources reserved for it — the shedder's promise is that admitted work will finish, not linger. On the shed path, a rejected request is turned around in microseconds with a 429; it consumed almost nothing, and the caller retries elsewhere or later. The ops strip names what you must watch: shed rate by tier (are you shedding only low-priority work, or has the threshold crept up to drop important work too?), queue latency (how long accepted work waits, the early warning of pressure), goodput (useful completed work, the number that shedding is meant to protect), and retry storms (are rejected callers backing off, or is Retry-After being ignored and amplifying load?).
A design nuance binds the pieces: the load signal and the queue bound must be chosen so the agent sheds before it is actually saturated, not after. If you wait until the worker pool is 100% busy and the queue is full to start rejecting, you have already accepted a backlog that will take time to drain, and latency has already spiked. Effective shedders reject at a threshold below true capacity — a queue that is 'full' at a depth chosen to keep tail latency bounded, not at the point of memory exhaustion. The bound is a latency budget expressed as a queue length, and setting it is the core tuning decision: too high and you admit work you cannot serve in time; too low and you shed work you could have handled.
End-to-end flow
Trace normal operation first. Requests arrive at a rate below the agent's capacity; admission control classifies each, the bounded queue stays shallow, workers pick up tasks promptly, and everything completes within its deadline. The load signal (queue depth) sits well under the shed threshold, so nothing is rejected. In this regime the shedder is invisible — it adds a classification step and a queue check, negligible overhead, and every caller gets its work done. This is the state the system is in almost all the time, and the shedder must be nearly free here or it is not worth having.
Now a demand spike: a burst of callers, or a fan-out from a planner, drives arrival rate above capacity. The queue begins to fill because workers cannot drain it as fast as it grows. Queue depth crosses the shed threshold, and admission control engages shedding — but selectively. Incoming background and speculative-retry requests get 429s with Retry-After; user-facing, first-attempt, high-tier requests are still admitted. The queue stops growing because the rejected work never enters it. The workers keep completing admitted tasks on time. From outside, the agent looks like it is running at exactly its capacity, shedding the overflow — which is precisely the designed behavior.
Watch what the 429 does on the caller side, because it is half the mechanism. A caller that receives 429 with Retry-After: 2s does not immediately re-send; it waits two seconds (ideally with jitter) before retrying, or it routes the task to a different agent, or it degrades its own output and moves on. This cooperative backoff is what keeps the rejection from being pointless: if callers ignored the signal and retried instantly, the shed requests would just come straight back, and the agent would spend all its time issuing 429s — shedding the same work over and over. The contract only works if both sides honor it, which is why a well-behaved A2A mesh treats Retry-After as binding, not advisory.
The performance character is the plateau. As arrival rate climbs from below capacity to far above it, accepted throughput rises to the agent's real ceiling and then stays flat — the shedder holds it there, dropping everything above the line. Latency of accepted work stays bounded because the queue never grows past its cap. Goodput (useful completed work) tracks accepted throughput rather than collapsing, because admitted work completes instead of timing out. Contrast the no-shedder curve: accepted throughput would rise, peak, and then fall as overload sets in, while latency runs to infinity. The shedder converts a collapsing curve into a flat plateau, and that plateau is the entire value proposition — predictable behavior no matter how far demand overshoots.
Consider the stress case that tests the shedder itself: a retry storm caused by the shedder's own rejections. Suppose a large fan-out sends a thousand tasks, the agent sheds six hundred of them, and every rejected caller retries at the same moment the Retry-After expires — a synchronized wave of retries slams the agent in one burst, overloading it again just as it was recovering. This is the shedder amplifying rather than damping load. The mitigations are decorrelated jitter on Retry-After (spread the retries across a window so they do not synchronize), a retry budget on the caller side (cap total retries so a persistently-shed task is eventually abandoned rather than retried forever), and circuit-breaking (a caller that keeps getting 429s from an agent stops sending to it entirely for a while). Without these, shedding can create the very metastable overload it was meant to prevent — which is why the caller-side contract is as much a part of the architecture as the server-side admission control.