Unavailable is a different problem than overloaded
An agent's best tool can be missing for reasons that have nothing to do with load: a third-party API is down, a permission scope was revoked, a specialist model was deprecated, a rate limit tripped for an unrelated tenant sharing the same key. This is a different failure than the one load shedding solves -- load shedding is about admission control and backpressure under too much traffic; this is about what an agent does when a specific capability it planned to use simply isn't there, at any traffic level.
The naive response -- catch the error, quietly substitute a worse capability, keep going -- is the one that causes the most damage, because it produces a response that looks exactly as confident as one produced by the intended, better capability. This article is about building the fallback chain as an explicit, ranked, disclosed decision rather than a silent exception handler.
The chain has to be designed, not discovered at runtime
A fallback chain is an ordered list, decided ahead of time, of what an agent tries when its first choice for a given capability isn't available: specialist tool, then a more general tool that covers a subset of the same ground, then an explicit request for human input, in that order, for a given capability slot. Discovering the chain at runtime -- catching whatever exception fires and reacting ad hoc -- produces inconsistent behavior across identical failures, because which fallback gets used ends up depending on incidental code paths rather than a considered ranking.
Ranking the chain requires answering one question honestly for each step down: what does this fallback lose relative to the step above it? A specialist code-search tool falling back to a generic grep loses semantic matching but keeps completeness; an agent falling back from a fine-tuned classification model to a general-purpose LLM prompt loses calibrated confidence but keeps rough correctness. Write that cost down per fallback level -- it's the input to the disclosure step later, and skipping it is how a team ends up with a fallback chain nobody can explain the tradeoffs of after the fact.
capability: "extract structured data from this invoice"
chain:
1. specialist-invoice-parser (cost if unavailable: none, best case)
2. general-document-ocr + LLM-extraction
(cost: lower field-level accuracy,
especially on non-standard layouts)
3. ask human to enter manually (cost: latency, but zero silent-error risk)
Why silent fallback is the dangerous default
The core risk isn't that a fallback capability is worse -- sometimes worse-but-available is the right call. The risk is that a response produced by the third-ranked fallback is, on the surface, indistinguishable from one produced by the first-ranked primary, and a user or a downstream system has no signal to treat it with appropriate skepticism. This is the same shape of problem as a system returning stale cached data during an outage without marking it as stale: technically available, silently wrong in a way nothing points at.
Concretely: an agent whose specialist data-extraction tool is down and falls back to a general LLM guess should not return that guess formatted identically to a verified extraction. If the caller (human or another agent in a pipeline) can't tell the two apart, every downstream decision built on that response inherits an invisible confidence gap. This is precisely the failure mode explored from the security angle in confused-deputy attacks -- there the ambiguity is exploited by an attacker; here it's self-inflicted by an agent's own fallback logic, but the underlying problem (a response that doesn't honestly represent its own provenance) is the same shape.
Signaling degraded capability, not just handling it
A well-designed fallback response carries three things a silent one doesn't: which capability actually produced this result, what's lost relative to the primary path, and whether the caller should treat the result as provisional. This doesn't need to be intrusive in every context -- a human-facing chat response can surface it as a short caveat; an agent-to-agent response in a pipeline should carry it as a structured field the receiving agent can act on programmatically (route to a review queue, lower a confidence score, trigger a retry later rather than persisting the result as final).
The design principle: degraded-capability disclosure should be as automatic and hard to skip as error handling itself, not an opt-in the author of a specific tool call has to remember. Wiring the fallback chain through one shared mechanism (rather than each tool call site implementing its own try/except with its own ad hoc fallback) is what makes the disclosure consistent instead of dependent on whichever engineer wrote that particular call site remembering to add it.
When to stop falling back and ask a human
The chain has to have a floor. Continuing to fall back through progressively worse capabilities rather than stopping and asking a human is how an agent produces a confidently wrong answer instead of an honest "I can't do this reliably right now" -- and a wrong answer that looks complete is worse than an incomplete one that's honest about why, especially for anything downstream that will act on it without a human in the loop.
A reasonable floor: if the gap between the primary capability and the next available fallback crosses a threshold the task's stakes don't tolerate (a financial calculation, a destructive action, anything the agent can't verify by another means), the chain terminates at "ask a human" rather than at the next-worse automated option, regardless of how many automated fallbacks remain in the list. This is a deliberate design decision made at chain-authoring time, not a runtime judgment call the agent invents under pressure -- the same reasoning that explicit state-machine design favors over ad hoc control flow generally: decide the hard cases in advance, when there's time to think about them, not live in the failure.
Design the fallback chain explicitly and rank it by what each step down actually loses -- don't let it be discovered ad hoc by whatever exception handler happens to fire. And never let a fallback response look identical to a primary one: disclose which capability actually produced a result and whether it's provisional, through one shared mechanism rather than leaving disclosure to whichever call site remembers to add it. A chain without a floor that stops and asks a human for high-stakes gaps isn't graceful degradation -- it's a confident wrong answer wearing a working one's clothes.