The event queue
Each active session has a bounded event queue. Model responses, tool results, and callback triggers are all enqueued. The loop drains one event at a time — sequential per session, parallel across sessions.
Advertisement
Ordering guarantees
Within one session, order is strict. If tool A completes before tool B was dispatched, A's result is processed first. This matters when you build streaming tools — order-out matches order-in.
Advertisement
Threading model
Under the hood the runtime uses a virtual thread per session (JVM 21+). Blocking calls inside a tool don't stall other sessions — each has its own carrier. Runtime code paths themselves are non-blocking.
// Tools can block — that's OK
public class SlowTool implements Tool {
@Override
public Object call(Map args) {
return httpClient.send(request, BodyHandlers.ofString());
// blocking is fine — virtual thread yields
}
}