Containing a Compromised Agent: Quarantine, Credential Revocation, and Taint Tracing

Assume one agent will go wrong

In a multi-agent system, one agent will eventually behave badly. Its memory may be poisoned, a dependency may be compromised, a model update may change behavior, or a partner agent may be taken over. The goal is not only to stop it but to know what it touched and to keep the rest of the system working. The site's agent kill switch guide covers stop levels, out-of-band control, and drills. This chapter focuses on what makes containment precise rather than blunt: per-agent identity, quarantine that isolates one agent, credential revocation that actually stops in-flight work, and taint tracing through the outputs the agent produced.

A concrete failure: the research agent that started giving orders

Northstar's research agent has a long-term memory store. Over two days, crafted support tickets lead it to store a note claiming that the operations team approved automatic refunds for a list of order numbers. From then on, its summaries to the concierge include lines such as "refund pre-approved". Several refunds are issued before a reviewer notices a pattern.

Containment is slow for structural reasons. All three agents share one service account, so revoking it would stop the whole assistant. The research agent's outputs are not labeled with their origin, so the team cannot easily list which summaries, memory entries, and tasks it influenced. And there is no way to pause one agent while keeping the others running. The team stops everything, losing a day of service, and still has to reconstruct the affected refunds manually from logs.

Architecture: identities, quarantine levels, and a provenance graph

Per-agent identity is the precondition for everything else. If each agent obtains its own credentials through a broker, as described in the delegated identity chapter, revoking one agent is a policy change at the broker. Pair it with short token lifetimes so that in-flight tasks fail at the next exchange rather than continuing for an hour.

Quarantine is a graduated state of the agent, enforced outside the agent. Observe adds sampling and review. Restrict removes write tools. Read-only removes all tools that change state and blocks messages to other agents except through review. Isolate removes the agent from routing entirely, and the orchestrator falls back to a reduced capability or a human queue.

Implementation: quarantine and taint tracing

Illustrative python; adapt the persistence and identity adapters to your service.

QUARANTINE_TOOLS = {
    "observe": None,
    "restrict": {"search_docs", "read_order"},
    "read_only": {"search_docs"},
    "isolate": set(),
}

def allowed_tools(agent_id, registry, requested):
    level = registry.quarantine_level(agent_id)
    allowed = QUARANTINE_TOOLS.get(level)
    return requested if allowed is None else [t for t in requested if t in allowed]

def tainted_since(graph, agent_id, since):
    """Items produced by the agent after `since`, plus everything derived from them."""
    frontier = list(graph.produced_by(agent_id, since))
    tainted = set()
    while frontier:
        item = frontier.pop()
        if item in tainted:
            continue
        tainted.add(item)
        frontier.extend(graph.derived_from(item))
    return tainted

Enforce allowed_tools in the orchestration layer and in a plugin so that a quarantined agent cannot reach tools even if its own configuration is unchanged. In ADK, a plugin registered on the runner applies across agents, which is the right place for a global quarantine policy; keep the authoritative quarantine state in a service the agent cannot write. The provenance graph is a set of edges from each produced item to its inputs, written by your framework adapters, not by the model. Taint tracing is then a graph traversal from the agent's outputs after the suspected start time.

Signals that justify quarantine

Quarantine decisions need evidence that is independent of the agent's own account of what it did. Useful signals compare what the agent attempted with what policy allows: a rise in denied tool calls, requests for scopes outside the agent's normal profile, attempts to delegate to agents it has never called, and outputs that assert approvals or permissions that the policy service has no record of. Changes in volume matter too, such as a sudden increase in memory writes or in messages to other agents.

Each signal has innocent explanations, including a new feature, a prompt change, or a busy day, so wire signals to graduated responses. A single anomaly might move the agent to observe and sample its outputs for review. A confirmed policy assertion with no backing record, or an attempted call to a forbidden tool, justifies restrict immediately. Because restrict and read-only are reversible and leave the rest of the system running, the threshold for using them can be lower than the threshold for a full stop.

Keep detection logic and quarantine state outside the agent's reach. An agent should not be able to read its own detection thresholds or modify its quarantine level, and a compromised agent's logs should be cross-checked against the authoritative records of the services it called.

Response sequence

When a detection fires, run a fixed sequence. Move the agent to restrict or read-only immediately; this is reversible and limits harm while you investigate. Revoke the agent's outstanding tokens at the broker and invalidate its sessions. Compute the taint set from the earliest plausible start time, not the detection time, and freeze tainted items: memory entries are hidden from retrieval, pending tasks are paused, and outgoing messages are held. Review effects in the taint set, such as the refunds in the example, using the authoritative ledgers rather than agent narratives. Then decide on recovery: purge or restore memory from a snapshot, rebuild the agent from a known good version, and return it through observe before full service.

NIST's incident response guidance, SP 800-61 Revision 3, frames incident response as part of overall cybersecurity risk management aligned with CSF 2.0; apply that program to agents rather than inventing a separate one.

Tradeoffs and worked case

Per-agent identities increase the number of credentials and policies to manage; automate issuance and keep the policy in code. Provenance graphs add write volume to every message; store compact edges and expire them with the data they describe. Taint tracing from an early start time can freeze many legitimate items; review in priority order by effect type, money and data exports first.

Replay the research agent case with the new design. The anomaly detector notices a spike in summaries containing approval language not backed by the policy service. The research agent moves to read-only, losing its ability to message the concierge directly. Its tokens are revoked. The taint set from the first suspicious memory write contains the poisoned memory entry, 46 summaries, and 9 refund proposals, of which 4 were committed. The committed refunds are reviewed against the ledger and flagged for recovery. The concierge and operations agents keep serving customers, with research questions routed to a reduced search path. After the memory store is restored from a snapshot and the poisoned tickets are removed, the research agent returns in observe mode. The figures belong to this illustrative scenario.

Verification steps

Drill containment regularly with synthetic incidents. Assert that moving an agent to each quarantine level removes exactly the expected tools, that revoked tokens fail at the next call, that taint tracing on a seeded graph returns the expected set, and that the rest of the system continues to serve. Measure time from detection to restrict, and from restrict to a complete taint set. The delegation boundaries lab includes a compromised research agent scenario so you can see which controls bound its effect.

Practice and continue

Open the Delegation boundaries: trace privileges across three agents lab to change the assumptions with deterministic synthetic data and inspect each decision. This chapter belongs to the Multi-Agent Security group of Practical LLM Security, which builds on the 25-chapter abuse and fraud prevention series.

• Sandboxing Agent-Generated Code in Multi-Agent Pipelines: Executors, State, and Untrusted Output

Sources and implementation scope

Primary references checked October 16, 2026. Framework details follow the linked documentation and, for ADK, the google-adk 2.12.0 package source; pin and test the version your application uses. The Northstar scenarios, policies, figures, and code are original illustrative designs, not measured production results or complete integrations.

• NIST SP 800-61 Rev. 3: incident response
• RFC 9700: OAuth 2.0 Security Best Current Practice
• ADK safety and security
• OWASP logging
• OWASP excessive agency, 2025 edition