The event queue

Each active session has a bounded event queue. Model responses, tool results, and callback triggers are all enqueued. The loop drains one event at a time — sequential per session, parallel across sessions.

Advertisement

Ordering guarantees

Within one session, order is strict. If tool A completes before tool B was dispatched, A's result is processed first. This matters when you build streaming tools — order-out matches order-in.

Advertisement

Threading model

Under the hood the runtime uses a virtual thread per session (JVM 21+). Blocking calls inside a tool don't stall other sessions — each has its own carrier. Runtime code paths themselves are non-blocking.

// Tools can block — that's OK
public class SlowTool implements Tool {
  @Override
  public Object call(Map args) {
    return httpClient.send(request, BodyHandlers.ofString());
    // blocking is fine — virtual thread yields
  }
}