Practical technical guides / Agent-era architecture

Resume Failed Agent Map Steps Without Repeating Completed Work

Running one agent child per list item forces four decisions: fail fast or collect, how many at once, what an input request pauses, and what survives a crash. A model and timelines for each.

By Sandeep Belgavi · Published 2026-11-13

Running one agent child per list item forces four decisions: fail fast or collect, how many at once, what an input request pauses, and what survives a crash. A model and timelines for each.

One step, many items

A large share of agent work is the same job applied to every element of a list: summarise each open help-desk case, classify each returned book, draft a reply for each overdue order. Workflow engines have long called this a map, a fan-out or a for-each step. Agent workflows bring it back with a twist. Each item is now a model call, or a small agent run with tools, so items are slow, cost money, fail for reasons that have nothing to do with the code, and sometimes stop to ask a person a question.

ADK Java 1.14.0, released on 2 November 2026, added a node for exactly this shape. Its ParallelWorkers.parallelNode(node, maxParallelWorkers) wraps a workflow node so it runs once per item of a list input. The class is marked @Experimental. Its Javadoc states the rules plainly: items start in input order, at most maxParallelWorkers at a time; each item runs as a child on its own branch; the node outputs the results as a list in input order; a failed item cancels the others and fails the node; and an item that asks for input stops queued items from starting. The constructor also marks the node to re-run on resume, and a comment in the source notes that items get the same generated id on resume because they subscribe in input order.

Those few rules are a design. Each one answers a question you have to answer for any map step, whatever engine you use. This guide walks through the questions with a worked example, a model you can run, and an interactive lab.

The worked example

Belgavi's Book Store, a fictional shop, keeps a help-desk queue. A nightly workflow takes eight open cases, CASE-5101 to CASE-5108, and asks an agent to summarise each one for the morning shift. In the model the summaries take 4, 7, 3, 9, 5, 6, 2 and 8 seconds. The numbers are synthetic, chosen so the schedule is easy to follow; real model calls vary far more.

With three workers, the schedule is deterministic. At t=0 the first three cases start. CASE-5103 finishes at 3 s and CASE-5104 takes its slot; CASE-5101 finishes at 4 s and CASE-5105 starts; and so on until CASE-5108, which started at 11 s, finishes at 19 s. The step completes at t=19 s after eight item starts. With one worker it takes 44 s, the sum. With eight workers it takes 9 s, the longest item. That arithmetic is the first lesson: the worker count trades wall-clock time against concurrent load on the model provider and on the tools the items call, and the lower bound is always the slowest item.

Question one: what does a failure mean?

Suppose the summary of CASE-5104 fails: the model returns something the output check rejects, or a tool times out. There are two defensible answers.

Fail fast treats the list as one unit of work. The first failure cancels the running items, starts nothing else and fails the step. This is what ParallelWorkers documents. In the example, CASE-5104 fails at t=12 s. At that moment CASE-5106 has run for 5 seconds and CASE-5108 for 1 second; both are cancelled, and those 6 seconds of model time are discarded. CASE-5101, 5102, 5103, 5105 and 5107 finished earlier and have results; the step as a whole has none.

Collect treats each item as independent. The failure becomes that item's result, the other items continue, and the step completes with errors at t=19 s, with seven summaries and one error.

Neither is right in general. Fail fast fits when the items feed a single decision: if one supplier quote in a set of five cannot be read, comparing the other four may mislead. Collect fits when items are independent deliverables, as nightly case summaries are; one bad case should not cost the shift the other seven. The mistake is to inherit whichever policy the engine defaults to without deciding. If your engine only offers fail fast, you can get collect by making the child catch its own error and return a typed failure value instead of raising.

def summarise_or_error(case):
    try:
        return {"case": case.key, "status": "ok", "summary": summarise(case)}
    except (ModelOutputRejected, ToolTimeout) as e:
        return {"case": case.key, "status": "error", "reason": type(e).__name__}

This keeps the failure visible and countable, which matters later: a step that quietly returns seven results out of eight looks like success to anything downstream that only checks the length was non-zero.

Question two: how many at once?

The worker bound is a capacity decision, not a performance knob. Each concurrent item holds a slot of the model provider's rate limit, a connection to each tool server, and memory for its context. Set it from the scarcest of those, not from the number of items. A list of 400 cases with an unbounded map is 400 simultaneous model calls, which a rate limit will turn into 429 responses and retries that make the whole step slower than a bound of eight would have been.

Starting items in input order has a quieter benefit: it makes the schedule reproducible, which makes the trace readable and the resume behaviour predictable.

Question three: what happens when an item needs a person?

Agent items can pause. A refund summary may need a staff member to confirm an amount before the agent can continue. ParallelWorkers' rule is that such an item stops queued items from starting. In the model, if CASE-5103 asks for input when it finishes at t=3 s, the running CASE-5101 and CASE-5102 complete at 4 s and 7 s, the step pauses at t=7 s, and five cases never start.

That is a conservative and sensible default. Starting new work while a question is outstanding risks a queue of half-finished items all waiting on the same person, and the answer to the question may change how the remaining items should run. If your items are truly independent, the alternative is to let the paused item park itself and keep the others flowing; but then the step needs a way to report partial completion and to resume only the parked items, which is the next question.

Question four: what survives a crash?

A long map step will eventually be interrupted: a deploy, an evicted pod, a lost lease. What happens on resume depends on whether each item's result was durably recorded when it finished.

In the model, a crash at t=12 s finds six cases finished and two running. With per-item checkpoints, the resume skips the six, starts the two that were lost, and completes at t=20 s: ten item starts in total. Without checkpoints, the resume runs all eight again from t=12 s and completes at t=31 s: sixteen item starts, and six cases summarised twice.

Re-running a summary costs money. Re-running an item that sent an email, issued a refund or updated a help-desk case does something worse. So checkpoints are necessary but not sufficient. The rule that holds up is: record each item's result durably when it finishes, and give each item's side effects an idempotency key derived from the step and the item, not from the attempt.

def item_key(workflow_run_id: str, step: str, case_key: str) -> str:
    return f"{workflow_run_id}:{step}:{case_key}"

The ParallelWorkers source comment implies that an item's identity on resume comes from its position: items get the same generated id because they subscribe in input order. Position is stable only while the input list is stable. If the list is recomputed on resume, from a query whose results changed overnight, positions shift and a position-keyed item may resume as the wrong case. Key by a business identifier, such as the case number, and freeze the list in the step's input when it starts.

What stays deterministic

The map itself should contain no model. The list to process, its order, the worker bound, the failure policy, the pause rule, the checkpoint and the idempotency key are plain code and configuration, versioned and reviewed like any other. The model runs inside each item, behind a contract: an input schema, an output schema and a validator that turns a bad output into a typed failure. A planner agent may propose which cases to include, but the step should record the frozen list it was given and refuse to grow it while running.

Migrating an existing batch job

Many teams already have a nightly batch that loops over records and calls a rule engine or a template. Moving it to agent items is a sequence of small changes. First, freeze the input list at the start and store it with the run. Second, add a per-item result record and make the loop skip items that have one. Third, put an idempotency key on every side effect the item can cause. Fourth, choose and document the failure policy. Only then replace the body of the item with a model call, starting with a shadow run that writes results somewhere harmless and compares them with the old output. Keep the old path for items the model declines or fails validation on.

When you do not need an agent here

If the per-item work is a lookup, a format conversion or a rule that fits in a decision table, a plain map over deterministic code is faster, cheaper and testable to the last case. Agents earn their place in a map only when each item needs judgement over unstructured input, such as reading a free-text complaint. Even then, ask whether one model call over a small batch of items would do: a single call that summarises eight short cases can be cheaper than eight calls, at the cost of coupling their failures.

Failure modes to test for

  • A step reports success with fewer results than inputs, because errors were dropped rather than recorded.
  • Cancellation is requested but the child keeps running, because the model client ignores it. Check that a cancelled item's model stream and tool calls actually stop.
  • A resume re-runs completed items because results were held only in memory.
  • Items are keyed by position and the input list changed between attempts.
  • An unbounded worker count turns one step into a rate-limit storm.

Verify it

Run the companion model and compare its timelines with the numbers above, then try your own durations in the lab. In your own engine, write four tests with a scripted fake model: one item fails under each policy; one item asks for input; the process is killed mid-step and resumed; and the input query returns a different list on resume. Assert the final results, the count of item starts, and that no side effect happened twice.

Run the example yourself

The map-step model with fail-fast and collect policies, bounded workers, input pauses, a crash and a resume with and without per-item checkpoints, and nine tests for every timeline quoted here. Companion code on GitHub; its README lists the prerequisites, exact commands, expected output and limits.

Interactive experiment

Workflow map step: fail fast, collect, bounded workers and resume. Run eight synthetic cases through a map step: workers, fail fast or collect, an input request, checkpoints and a crash. Watch starts, waste and resume. Synthetic inputs; read its assumptions and limits before drawing conclusions.

Primary references and authorship

Sources read 2026-11-13. Original explanation and synthetic examples by Sandeep Belgavi. Views are my own and do not represent any employer, past or present.