Why architecture matters here
The architecture matters because agent workloads are the near-perfect case for virtual threads and the near-worst case for the platform-thread model they replace. An agent turn is a sequence of long, blocking waits punctuated by tiny bursts of CPU: call the model (wait), parse the response (microseconds), call a tool (wait), feed the result back (microseconds). With platform threads, every one of those waits pins a scarce OS thread doing nothing. Size the pool to the cores and you can serve only a handful of concurrent agents before requests queue; size it to the concurrency you want and the JVM strains under thousands of kernel threads, each with a large stack, competing for the scheduler. Virtual threads break the coupling between 'a task is in progress' and 'an OS thread is consumed', so concurrency is bounded by memory and downstream capacity rather than by thread count.
The second forcing function is code clarity, which is not a luxury in agent systems. Agent orchestration is already conceptually hard — plan, call, observe, re-plan, handle partial failures — and expressing it in callback chains or reactive streams multiplies the difficulty, scattering a single logical flow across lambdas and making stack traces useless. Virtual threads let the Java ADK express an agent turn as ordinary sequential code: a method that calls the model, calls a tool, and returns, with real stack traces, real try/finally, and real debugger stepping. The framework gets reactive-grade scalability while its authors and users write and read blocking-style code, which is where correctness lives.
The third reason is that the abstraction is not free of sharp edges, and the edges are exactly where an agent framework operates. A virtual thread's superpower — unmounting at a blocking call — is defeated when the thread is pinned: if it blocks while inside a synchronized block or during a native (JNI) call, the JVM cannot unmount it, so it holds its carrier hostage for the whole wait. A few pinned threads blocking on slow LLM calls can starve the small carrier pool and collapse the throughput of the entire process. Because agents make long blocking calls constantly, an ADK built on virtual threads must be scrupulous about never holding a monitor lock across an I/O call, replacing synchronized with ReentrantLock where blocking is possible, and understanding which libraries pin.
The fourth architectural payoff is cancellation and lifecycle, which agents need acutely. An agent turn may spawn several concurrent tool calls and must cancel the rest the moment one fails, or abandon the whole turn when the user disconnects or a deadline passes. Structured concurrency, the companion API to virtual threads, makes this tractable: child tasks are opened in a scope that owns them, and closing the scope guarantees every child is either joined or cancelled, so no tool call outlives the turn that started it. This turns the messy problem of 'clean up all the in-flight work when an agent gives up' into a lexically-scoped, exception-safe block, which is precisely the reliability property a long-running agent server must have to avoid leaking connections and runaway calls.
The architecture: every piece explained
Top row: from request to running code. An agent request — one turn, or one tool invocation within a turn — is handed to a virtual thread, which is cheap enough that the framework creates one per task without a pool and discards it when the task ends. The JVM scheduler (a ForkJoinPool by default) mounts the virtual thread onto a carrier — a real platform thread from a small pool whose size defaults to the number of processors. While the virtual thread is doing CPU work it occupies a carrier exactly like a normal thread; the difference only appears when it blocks. The carrier pool is deliberately small because carriers should almost never sit idle: whenever a virtual thread blocks, its carrier is handed to another runnable virtual thread.
Middle row: the mount/unmount dance that makes it all work. When the virtual thread hits a blocking I/O call — the LLM request, a tool's HTTP call, a JDBC query — the JVM intercepts the block and parks the virtual thread, unmounting it from its carrier and saving its continuation (its stack, captured on the heap). The carrier is now free to run other virtual threads, so a thousand agents blocked on the model share a handful of carriers. When the I/O completes, the runtime marks the virtual thread runnable and the scheduler remounts it — possibly on a different carrier than before — and execution resumes on the line after the blocking call, with the full local state intact. From the code's point of view nothing happened except that a method returned; from the machine's point of view, no OS thread was ever blocked.
Bottom row: coordination and hazards. Structured concurrency is how an agent fans out safely: a scope (StructuredTaskScope) forks several child virtual threads — parallel tool calls, or racing two model providers — and the scope's join gathers them, propagating the first failure and cancelling the siblings, so the children's lifetimes are strictly nested inside the parent's. Pinning hazards are the counterweight: blocking inside a synchronized block or a native call prevents unmounting, so the virtual thread holds its carrier for the entire wait, and enough pinned waits starve the carrier pool. The ops strip names the signals that matter — carrier utilization, time spent pinned, scope cancellation behavior, and the heap consumed by the continuations of parked virtual threads.
End-to-end flow
Trace one agent turn that calls the model and then fans out to two tools. A request arrives at the ADK's HTTP server, which is itself running on a virtual-thread-per-request executor, so the request already owns a virtual thread. The handler invokes the agent, which builds a prompt and calls the language model. That call blocks on the network for eight hundred milliseconds; the JVM parks the virtual thread and unmounts it, and the carrier immediately picks up another agent's runnable virtual thread. Meanwhile hundreds of other agents are parked on their own model calls, all sharing the same handful of carriers — the process is busy but its OS threads are never blocked.
The model returns a plan that requires two tool calls — a database lookup and an external API fetch — that are independent, so the agent runs them in parallel. It opens a structured-concurrency scope and forks a child virtual thread for each tool. Each child blocks on its I/O and parks; the parent virtual thread parks on the scope's join, waiting for both children. When the database lookup returns in forty milliseconds, its child virtual thread remounts, finishes, and records its result in the scope. When the API fetch returns two hundred milliseconds later, its child finishes too, the join completes, and the parent resumes with both results. The whole fan-out consumed three virtual threads for a couple hundred milliseconds and zero blocked carriers — the carriers ran other agents' work throughout.
Now a failure injects itself. On a different turn, one of the two tool calls throws — the API returns a 500. Inside the structured scope, the child's exception is captured, the scope cancels the sibling child (interrupting its in-flight database call so it stops wasting a downstream connection), and the join rethrows into the parent. The parent's ordinary try/catch handles it: it either retries, degrades gracefully, or returns an error to the user. Crucially, when the scope's block exits, structured concurrency guarantees that neither child virtual thread is still running — there is no orphaned tool call continuing in the background against a turn that has already failed. This is the lifecycle guarantee that makes fan-out safe.
Finally, a pathological turn shows the sharp edge. An older library the agent calls for a tool grabs a synchronized monitor and then makes a blocking HTTP call inside it. The virtual thread running that tool is pinned: the JVM cannot unmount it, so it holds its carrier for the full duration of the HTTP wait. Under light load nobody notices. Under a burst of concurrent agents all routing through that same library, pinned virtual threads occupy every carrier in the small pool, and the process stalls — new agents cannot get a carrier to run on even though nothing is CPU-bound. The operational fix is to find the pinning (the JVM can emit an event when a thread blocks while pinned), replace the synchronized with a ReentrantLock that does not pin, or move the blocking call outside the lock. Once unpinned, the same code parks cleanly and the stall disappears — the cycle of mount, block, unmount, resume runs as designed.