Why architecture matters here
The architecture matters because blocking an agent on long work does not scale and does not survive. An agent runtime typically handles many concurrent sessions; a tool call that blocks for ten minutes ties up that session's execution slot and any resources it holds for the whole time, so a handful of long calls can starve the runtime. Worse, real deployments restart — deploys, crashes, autoscaling — and an in-memory blocking call does not survive a restart: the work is either lost or orphaned, and the agent has no memory it was waiting. Long-running tools decouple the agent's liveness from the work's duration: the agent yields, its state is durably saved, and the work proceeds independently, so neither a long duration nor a restart destroys the interaction.
The second forcing function is human-in-the-loop, which is fundamentally a long-running operation. Many agent workflows must pause for a person — approve this refund, review this draft, authorize this deployment — and a human might respond in seconds or not until tomorrow. Modeling that approval as a long-running tool is exactly the right fit: the tool 'starts' the approval request (posts it to a queue, sends a notification), returns a handle, and the agent pauses with its state saved; when the human acts, a callback resumes the agent with the decision. Without a long-running-tool mechanism, human approval forces awkward workarounds — busy-waiting, external orchestration outside the agent — whereas with it, a human step is just another tool call the agent awaits.
The third reason is that resuming correctly across a pause is subtle, and the architecture has to guarantee it. When the agent yields on a long-running tool, its conversation state — the messages, the pending tool call, the model's context — must be persisted so that, whenever completion arrives, the runtime can reconstruct exactly where it left off and feed the result into the right place. If the resume loses context, or fires twice, or attaches the result to the wrong pending call, the agent's reasoning breaks. So the session store, the operation handle that ties a completion back to a specific paused turn, and the resume logic form a tight contract: one completion resumes one paused turn, exactly once, with the full prior context intact.
A fourth reason is bounding and cleanup. Async work that the agent has launched can fail, hang, or never complete — the external pipeline errors out, the human never responds, the callback is lost. If the agent simply waits forever, the session leaks: it sits pending indefinitely, and the launched work may run orphaned. A robust long-running-tool architecture therefore includes a timeout on the wait and a cancellation path: after a bounded time the runtime resumes the agent with a timeout result so it can decide what to do (retry, escalate, give up), and it signals the underlying operation to cancel so no orphaned job keeps running. This mirrors distributed timeout handling, and it is what keeps a fleet of long-lived agents from accumulating a graveyard of stuck sessions and abandoned jobs.
The architecture: every piece explained
Top row: launching the work. An agent turn begins when the model, reasoning about the task, decides to call a tool. The long-running tool is a tool declared as asynchronous: rather than running the work inline, it starts an async job — submitting to a pipeline, posting an approval request, launching a batch — and immediately returns an operation handle (an operation id plus a way to check status) instead of a final result. The external work is whatever actually takes time: a data job, a rendering pipeline, a human approval, a third-party long call. The handle is the thread that will later tie the eventual completion back to this specific invocation.
Middle row: surviving the wait and coming back. On a long-running call the runtime performs a session pause — the agent yields rather than blocking, and its conversation state is saved. State persistence writes that session to a durable store (keyed by session and the pending operation) so it outlives the process and can be reconstructed later. The poll / callback mechanism is how completion is detected: the runtime either polls the operation's status endpoint on an interval, or the external work calls back (a webhook) when it finishes. When completion arrives, resume turn loads the persisted session, feeds the operation's result into the model as the tool's return value, and the agent continues reasoning from exactly where it paused.
Bottom row: the human path and the bounds. Human-in-the-loop is a first-class use of this machinery — an approval or review step is modeled as a long-running tool whose 'completion' is a person's decision arriving via callback, so the agent awaits a human exactly as it awaits a pipeline. Timeout + cancel bounds the wait: a deadline resumes the agent with a timeout outcome if completion never comes, and a cancellation signal stops the underlying operation so it does not run orphaned. The ops strip names what to watch across a fleet of long-lived agents: the number of pending operations, completion latency, orphaned jobs, and resume failures.
End-to-end flow
Trace a data-pipeline tool through the full lifecycle. A user asks an agent to 'run the nightly enrichment job and summarize the results.' The model calls the long-running run_enrichment tool. The tool submits the job to the pipeline service, gets back a job id, and returns an operation handle — not results — to the runtime. The runtime pauses the session: it persists the conversation, the pending tool call, and the operation id to the session store, and releases the execution slot. From the runtime's perspective this session is now 'waiting on operation X'; it is consuming no compute, and it would survive a full restart because everything needed to resume is durably stored.
Minutes later the pipeline finishes and posts a webhook to the runtime: 'operation X complete, here is the summary manifest.' The runtime looks up which paused session is waiting on operation X, loads that session from the store, and resumes the agent turn by feeding the manifest in as the return value of the original run_enrichment call. The model, now with its full prior context restored plus the tool result, continues: it reads the manifest and writes the summary the user asked for. From the model's point of view, the tool simply took a while to return — the pause, the persistence, and the resume were invisible plumbing that made a multi-minute operation look like an ordinary tool call.
Now the human-in-the-loop variant. The task is 'issue a refund if the claim is valid,' and policy requires human sign-off above a threshold. The agent reasons the refund is warranted and calls the long-running request_approval tool, which posts the request to an approvals queue with the claim details and returns a handle. The session pauses. A human reviewer sees the request an hour later and approves it; the approval system fires a callback with the decision. The runtime resumes the agent, feeding in 'approved,' and the agent proceeds to issue the refund. Had the reviewer rejected it, the same mechanism would resume the agent with 'rejected' and it would decline instead. A day-long human gate and a minutes-long pipeline used exactly the same architecture.
Finally the unhappy paths the bounds exist for. Suppose the pipeline job hangs and never posts its webhook. The wait's timeout fires: the runtime resumes the paused session with a timeout result, and the agent can retry the job, escalate to a human, or report failure to the user — rather than leaving the session pending forever. Alongside, the runtime signals the pipeline to cancel the orphaned job so it stops consuming resources. And the resume path is guarded for exactly-once: if a flaky webhook fires the completion callback twice, the runtime recognizes the operation is already resolved (the session is no longer waiting on that operation) and ignores the duplicate, so the agent turn resumes once, not twice. These guards — timeout, cancel, dedupe on the operation handle — are what make long-running tools reliable rather than a source of stuck and double-executed sessions.
Notice how the same three primitives — persist, resume, guard — carry every one of these scenarios, whether the wait is seconds or days and whether the completion comes from a pipeline, a human, or a third-party service. Persistence is what makes the pause durable, so a restart mid-wait loses nothing; the resume contract is what reconstructs the exact paused turn and feeds the result into the right place; and the guards — timeout, cancel, dedupe on the operation handle — are what keep the unhappy paths from turning into leaked sessions or double-executed actions. An agent runtime that gets these three right can treat 'wait for a long thing' as an ordinary, reliable capability, which is precisely what unlocks durable, multi-step, human-in-the-loop workflows that a purely synchronous tool model could never express.