A2A Task Lifecycle Under Failure: Streaming, Push Notifications and Reconnects

The task is the unit of truth, not the connection

Everything in this guide follows from one sentence in the A2A specification: when an agent serves several streams for the same task, "the task lifecycle is independent of any individual stream's lifecycle." A2A (this guide follows specification version 1.0.0, the latest released version at the time of writing, with the 1.0.1 patch notes checked) models long-running work as a Task object that lives on the remote agent, has a server-generated ID, belongs to a contextId, and moves through a defined set of states. A stream, a webhook delivery or a poll is only a way of observing that task.

Production failures in multi-agent systems mostly come from clients that confuse the two. They treat a broken stream as a failed task and start the work again, treat a missing webhook as "still running", or treat a 404 from a restarted agent as a transient error. The fixes are all available in the protocol; they just have to be used deliberately. SDKs adopt specification versions at different speeds (1.0 renamed several fields and methods relative to 0.3, and clients must send the A2A-Version header), so check which version your client and agent SDKs implement before relying on any detail below.

States, and which ones can stall

The 1.0 data model defines these task states:

Terminal states are final: sending a message to a terminal task returns UnsupportedOperationError, and so does subscribing to one. The interrupted states are the dangerous ones operationally, because nothing fails loudly. A task waiting for input simply waits, and if the client never learned it was waiting, the work stalls until some timeout on the agent side turns it into a failure. In the lab, 15% of tasks pause for input part-way through and give up after 15 minutes without an answer, which is why a lost notification shows up as a missed input request rather than a lost result.

Blocking matters too. SendMessage is blocking by default in 1.0: unless returnImmediately is true, the call waits until the task is terminal or interrupted. A client that wraps a blocking call in a short HTTP timeout and retries on timeout is re-sending the same message, which leads straight to the next problem.

Streams break; reattach instead of resending

SendStreamingMessage opens a stream (Server-Sent Events in the JSON-RPC and HTTP+JSON bindings) that begins with the Task object, carries TaskStatusUpdateEvent and TaskArtifactUpdateEvent objects in order, and closes when the task reaches a terminal state. Version 1.0 removed the old final flag from status updates; the end of the stream is the signal. Streams break for ordinary reasons: proxy idle timeouts, mobile networks, deploys of the client itself.

The protocol's answer is SubscribeToTask (POST /tasks/{id}:subscribe in the HTTP binding). Its first event must be the current Task, which the spec says exists precisely to prevent losing information between a GetTask and a subscription. A client that reconnects this way loses nothing: it sees the current status and artifacts, then the rest of the events.

# after the stream for task t-81c2 drops (HTTP+JSON binding)
POST /tasks/t-81c2:subscribe HTTP/1.1
Host: agent.example.com
Authorization: Bearer ...
A2A-Version: 1.0

HTTP/1.1 200 OK
Content-Type: text/event-stream

data: {"task": {"id": "t-81c2", "contextId": "c-17", "status": {"state": "TASK_STATE_WORKING"}, "artifacts": [ ... ]}}

data: {"statusUpdate": {"taskId": "t-81c2", "contextId": "c-17", "status": {"state": "TASK_STATE_COMPLETED"}}}

# if the task already finished, subscribe returns UnsupportedOperationError: call GET /tasks/t-81c2

The common mistake is to call SendStreamingMessage again with the original message. Unless the agent deduplicates on messageId, which the spec allows but does not require ("Send Message operations MAY be idempotent"), that starts a second task while the first keeps running and commits its own side effects. The arithmetic is unpleasant: if streams drop on average every eight minutes and a task runs for twenty, the expected number of drops per task is 2.5, so a resend-on-drop client creates more duplicate tasks than real ones and may never see a long task finish at all, because each resend starts it from scratch. Mission one in the lab reproduces this with network flaps and asks for zero duplicates.

Push notifications: at least once, with an end to the retries

For server-to-server integrations and tasks that run for minutes or hours, holding a stream open is wasteful. The client registers a webhook with a TaskPushNotificationConfig, either inline in SendMessageConfiguration or through CreateTaskPushNotificationConfig, and the agent POSTs a StreamResponse (a task, message, status update or artifact update) when the task changes. The delivery contract in section 4.3.3 is deliberately minimal:

• Agents must attempt delivery at least once for each configured webhook, may retry with exponential backoff, should time out a webhook call after 10 to 30 seconds, and may stop after a configured number of consecutive failures.
• Clients must answer with a 2xx status, must validate the credentials the agent sends in the configured authentication header, should check that the task ID matches a task they created, and should process notifications idempotently because duplicates may occur.

Both halves create failure modes. Retries end, so a receiver outage longer than the retry schedule loses events. With a 5-second base and doubling, the cumulative wait after k retries is 5 × (2^k - 1) seconds: seven retries cover about 10.6 minutes and eight cover about 21. A 12-minute outage of the receiver therefore needs eight retries, and even then an input-required notification may arrive after the agent's input timeout. And delivery is at least once: if the receiver processes an event but its response is lost or slow, the agent retries and the same terminal event arrives twice. A handler that ships an order or posts a payment on every TASK_STATE_COMPLETED will do it twice unless it records the task ID and state it has already handled.

Push also has security obligations that belong in the same design review: the spec tells agents to validate webhook URLs against SSRF (reject private ranges, localhost and link-local addresses, use allowlists) and tells clients to use HTTPS, unique single-purpose tokens per configuration, and rate limiting.

Polling and reconciliation: the slow path that catches everything

GetTask works with every binding, through every firewall, and needs no inbound connectivity. Its costs are simple to compute. With a polling interval P, the client learns of a state change P/2 later on average and up to P later, and a task of duration D costs about D/P requests. For tasks averaging nine minutes, polling every 15 seconds is around 36 requests per task; polling every two minutes is under five, at the price of minute-scale latency.

The more useful role for polling is as a reconciliation loop behind a fast path. Streams and webhooks give low latency; a slow periodic check of every non-terminal task the client believes is in flight catches whatever the fast path dropped: an outage longer than the retry budget, a stream that died without a reconnect, or a task the agent no longer knows. In 1.0 this does not have to cost one request per task: ListTasks returns the caller's tasks sorted by last update, filtered by contextId, status and statusTimestampAfter, with cursor pagination, so a single paged call per interval can find every task that changed since the last check. The lab charges reconciliation per task, which overstates its cost; treat its request counts as an upper bound.

Mission two puts a webhook receiver behind a 12-minute outage and limits the client to two requests per task. Raising retries recovers most outcomes but still misses input requests whose notifications arrive too late; a ten-minute reconciliation poll catches them for well under one extra request per task.

When the agent forgets: restarts and lost tasks

The A2A specification describes behaviour, not storage. An agent that keeps tasks in process memory is conformant until it restarts, at which point every in-flight task disappears. Streams to it drop; SubscribeToTask and GetTask return TaskNotFoundError; and push notifications simply stop, because the agent no longer knows there is anything to notify about. A push-only client that relies on the agent to speak first will wait forever.

Two things fix this, on different sides of the boundary. If you operate the agent, persist tasks (state, history needed for follow-ups, artifacts and push configs) in a store that survives deploys, and make in-flight work resumable or at least fail it explicitly on startup so clients get a terminal state. If you only operate the client, you cannot assume the agent does any of that, so the client must own a list of tasks it expects to finish, check them periodically, and treat TaskNotFoundError for a task it created as "lost: resubmit or escalate", never as "not yet visible". Mission three restarts the agent at minute 60 with no persistence and asks you to recover every outcome without touching the agent; the three-star answer uses the least frequent reconciliation that still works.

Choosing the delivery mix

The three mechanisms are complementary, and the spec presents them that way. A reasonable default by situation:

Mission four turns on every failure at once (network flaps, a receiver outage, lost acknowledgements and an agent restart) and starts from 15-second polling that works but costs dozens of requests per task. Several configurations meet the goal; the cheapest combine a fast path, a ten- to fifteen-minute reconciliation and an idempotent handler.

Worked example: overnight catalog enrichment

Suppose a hypothetical orchestrator at Belgavi's Book Store delegates catalog enrichment to a remote A2A agent: 240 tasks over two hours, each running a few minutes to an hour (median six), 15% pausing once to ask which edition a record refers to. All numbers are from the lab's synthetic model.

1. Streaming with resend-on-drop during network flaps creates around 700 duplicate tasks and leaves about 6% of outcomes unknown, because long tasks keep restarting.
2. Streaming with SubscribeToTask removes every duplicate at a cost of about one extra request per task.
3. Push with one retry during a 12-minute receiver outage loses roughly one outcome in ten and several input requests.
4. Push plus a ten-minute reconciliation and an idempotent handler learns every outcome, survives the restart by resubmitting lost tasks, and uses under two requests per task, against more than 35 for 15-second polling.

The pattern generalises: decide which mechanism provides latency, which provides completeness, and where duplicates are absorbed.

Detecting lifecycle failures in production

Most of these failures are invisible in request metrics, because every individual call succeeds. Track the lifecycle instead:

• Tasks the client created that have no terminal state after their expected duration plus a margin: the orphan count.
• Time from the agent's status timestamp to the client's processing time, per mechanism, so you can see push delays during receiver incidents.
• Tasks in TASK_STATE_INPUT_REQUIRED or TASK_STATE_AUTH_REQUIRED by age; anything older than a few minutes is a missed notification.
• Duplicate deliveries absorbed by the idempotent handler, and new tasks created with a messageId the client has used before.
• TaskNotFoundError on tasks the client created, which means the agent lost state.

The common mistakes are re-sending messages on stream errors, treating webhooks as exactly once, setting retry budgets without comparing them with real outage durations and input timeouts, wrapping blocking SendMessage calls in short timeouts with automatic retry, and skipping reconciliation because "push is reliable". It is reliable at least once, for as long as the retries last and the agent remembers the task.

Run the lab: A2A tasks under failure: reattach, retry, reconcile

The lab simulates 240 delegated tasks over two hours against a remote A2A agent. The canvas shows sixteen of the longer tasks as horizontal bars: blue while working, amber while waiting for input, pink for duplicate tasks the client accidentally created. Green dots mark when the client learned the outcome, white crosses mark dropped streams and a question mark means the outcome was never learned. Choose how the client receives updates and how it reacts to failures, then inject flaps, an outage, lost acknowledgements or an agent restart. Each mission loads a broken setup.

Prefer a full-width view? Open the standalone lab. Your stars are saved in this browser and shown on the series hub.

What to do next

1. Search your A2A client for any code path that re-sends a message after a stream or timeout error, and replace it with SubscribeToTask or GetTask on the existing task ID.
2. Record every task ID your client creates in a durable table with its expected duration, and alert on tasks with no terminal state after that duration plus a margin.
3. Add a reconciliation loop using ListTasks with statusTimestampAfter, and treat TaskNotFoundError on your own tasks as lost work.
4. Make webhook handlers idempotent on task ID and state, validate the webhook credentials, and return 2xx only after the event is durably recorded.
5. Compare the agent's push retry schedule with your receiver's worst outage and with input-required timeouts.
6. Alert on tasks sitting in input-required or auth-required for more than a few minutes.

Sources and further reading

• A2A Protocol Specification (latest released version)
• A2A Protocol Specification v1.0.0
• A2A protocol repository: specification, a2a.proto and release notes
• A2A topic guide: streaming and asynchronous operations
• WHATWG HTML Living Standard: server-sent events

Related in this series:

• Retrying Agent Tool Calls Without Executing the Same Action Twice
• Supervisor or Swarm: When Multi-Agent Systems Actually Beat One Agent
• MCP Streamable HTTP in Production: Sessions, Resumption and Load Balancers

Diagrams, examples and the lab model are original educational material. Lab numbers are calculated by a synthetic model, not measured on real hardware or services.

Views are my own and do not represent any employer, past or present.