Parallel Tool Calls in Agent Loops: Dependency Classification, Partial Failure Semantics and Cancelling Siblings

The model asks for five things at once

Current function-calling APIs let a model emit more than one call per turn. The OpenAI function calling guide describes parallel function calling and a parallel_tool_calls setting that, when false, ensures exactly zero or one tool is called. The Gemini function calling guide distinguishes parallel function calling, for independent functions in a single turn, from compositional calling, where calls happen in sequence because one depends on another. Both leave the important part to you: the API delivers a list of calls, and the runtime decides whether they run concurrently, in what order, with what timeouts, and what goes back to the model when two of five fail.

That decision is a scheduling problem with side effects, not a concurrency convenience. The Agent Development Kit, which runs tool batches concurrently, shipped a fix on 12 October 2026 because a cancellation originating in one tool task left its siblings running; the error type for cancellation was not caught by a handler written for ordinary exceptions. A runtime that has not thought about these cases has the same bugs waiting.

Classify before you schedule

The classification relies on tool metadata that the model never sees: whether a tool reads or writes, which resource keys its arguments touch, and which backend it uses. Adding that metadata is the real work of supporting parallel calls. Without it the runtime has two choices, serialise everything or run everything concurrently, and both are wrong often enough to matter.

A scheduler in code

Illustrative python, checked for syntax; adapt names and storage to your system.

import asyncio

TOOLS = {   # metadata the model never sees
    "get_weather":   {"mode": "read",  "backend": "weather", "keys": lambda a: set()},
    "search_hotels": {"mode": "read",  "backend": "hotels",  "keys": lambda a: set()},
    "book_hotel":    {"mode": "write", "backend": "hotels",  "keys": lambda a: {("hotel", a.get("hotel_id"))}},
    "cancel_hotel":  {"mode": "write", "backend": "hotels",  "keys": lambda a: {("hotel", a.get("hotel_id"))}},
}
LIMITS = {"weather": 4, "hotels": 2}

def plan_waves(calls):
    """Group calls into waves that may run concurrently. Dependent or malformed calls are deferred."""
    waves, deferred, written = [], [], set()
    current, current_keys = [], set()
    for call in calls:
        meta = TOOLS[call["name"]]
        args = call["arguments"]
        if any(v in (None, "", "<pending>") for v in args.values()):
            deferred.append((call, "argument depends on a result that is not available yet"))
            continue
        keys = meta["keys"](args)
        conflict = keys & current_keys or (meta["mode"] == "write" and keys & written)
        if conflict and current:
            waves.append(current)
            current, current_keys = [], set()
        current.append(call)
        current_keys |= keys
        if meta["mode"] == "write":
            written |= keys
    if current:
        waves.append(current)
    return waves, deferred

async def run_calls(calls, execute, timeout=10.0):
    semaphores = {b: asyncio.Semaphore(n) for b, n in LIMITS.items()}
    results = {}

    async def one(call):
        async with semaphores[TOOLS[call["name"]]["backend"]]:
            try:
                value = await asyncio.wait_for(execute(call), timeout)
                return {"status": "ok", "result": value}
            except asyncio.TimeoutError:
                return {"status": "timeout", "retryable": True}
            except Exception as e:                      # tool error, not a runtime bug
                return {"status": "error", "error": str(e), "retryable": False}

    waves, deferred = plan_waves(calls)
    for wave in waves:
        outcomes = await asyncio.gather(*(one(c) for c in wave), return_exceptions=False)
        for call, outcome in zip(wave, outcomes):
            results[call["id"]] = outcome
    for call, why in deferred:
        results[call["id"]] = {"status": "deferred", "error": why, "retryable": True}
    assert set(results) == {c["id"] for c in calls}, "every call id must be answered"
    return results

Two properties are deliberate. The assertion at the end encodes the contract that every call id receives exactly one result, whatever happened. And errors inside a tool are converted into results rather than raised, so one failing call cannot abort the wave. Runtime failures such as cancellation are a different matter, handled next.

Partial failure: what the model should see

Return one result per call id, always, including for calls that were never run. Give each result a status the model can act on: ok, error, timeout, deferred or pending approval, plus a retryable flag. Never collapse five results into one error message; the model then re-issues calls that succeeded, which is how duplicate bookings happen. Never drop a result because another call failed; the model should learn that four of five worked and decide what to do about the fifth. The order of results does not matter as long as ids are present, and the result for a deferred call should say why it was deferred, so the model can issue it correctly in the next turn rather than guess.

Cancellation semantics

Cancellation arrives from two directions. From above, the run is cancelled by a user, a budget or a parent agent; every task in the wave should be cancelled, and the runtime must wait for them to finish cancelling before it reports, or tasks leak. From inside, one tool task is cancelled or raises a cancellation while its siblings continue; here the policy is a choice. Fail-fast cancels the siblings, which is right when the wave is a unit of work the model will redo. Let-finish lets siblings complete and reports them, which is right when they are independent reads whose results remain useful.

The ADK commit of 12 October 2026 is a compact illustration of the implementation trap: a handler written as except Exception does not catch asyncio.CancelledError, because in Python cancellation derives from BaseException, so the sibling-cancellation branch was never reached for a child-originated cancellation. Whichever policy you pick, test it with a cancellation that originates inside a tool, not only from the caller. Write tools deserve one more rule: a write that may already have taken effect must not be cancelled mid-flight unless it carries an idempotency key, or the model will retry a booking that exists.

Worked example: an itinerary agent

A travel agent model emits four calls in one turn: weather for city A, weather for city B, a hotel search in city B, and a booking for hotel h1 with the hotel id left as a placeholder because it expects to pick one from the search. The scheduler classifies the two weather calls and the search as independent reads and runs them concurrently, with the hotels backend limited to two concurrent calls. The booking has a placeholder argument and is deferred with the reason that it depends on a result not yet available. The response to the model is four entries: three ok results and one deferred, with ids. On the next turn the model books with a real id, which is now a single write and runs alone. Had the runtime executed the booking concurrently, it would have failed inside the booking service with an opaque error or, worse, booked a default.

Testing the runtime

• Fake tools with controllable latency, failures and cancellation points; assert that every call id is answered for every combination.
• A write that fails after taking effect, retried by the model: assert the idempotency key stops the second effect.
• A cancellation raised inside one tool: assert no sibling task is left running after the runtime returns.
• Two writes to the same resource key in one turn: assert they are serialised in emission order.
• Fifty read calls to one backend: assert concurrency never exceeds the configured limit.
• A property test: the set of result ids equals the set of call ids, for random mixes of the classes in the table.

Failure modes

• Duplicate calls in one turn. The OpenAI guide notes a model snapshot that sometimes included multiple calls for the same tool when parallel calls were enabled. Deduplicate identical name-and-argument pairs before scheduling, and answer both ids.
• Orphaned tasks. Exceptions caught too narrowly, as in the ADK fix, or a gather that is not awaited on error.
• Unbounded fan-out. Twenty parallel searches against a backend with a quota of five produce rate-limit errors the model cannot interpret. Use the per-backend semaphore.
• Oversized combined results. Five tool results in one turn can exceed the context budget together; apply the per-result budgets from sizing tool results for agent context.
• An approval-required action in a parallel wave. It must be held and reported as pending, not executed because its siblings were safe.

Practice and related reading

Further reading:

• Agent tool use: patterns and pitfalls
• The ReAct pattern revisited
• Sizing tool results for agent context
• Tool-calling reliability: timeouts, idempotency and compensation
• ADK workflow agents: sequential, parallel and loop

Sources and scope

Checked against these sources on 18 October 2026. Examples and figures are illustrative, not measurements.

• OpenAI API: function calling, including parallel function calling
• Gemini API: function calling, parallel and compositional calls
• Python documentation: asyncio coroutines and tasks
• google/adk-python commit 1ab2fe8: cancel sibling tools on child-originated cancellation (2026-10-12)