Evidence label: deterministic synthetic model. This page runs a small JavaScript model of an agent calling a side-effecting tool. Each synthetic action is sent with the chosen timeout, retries and backoff; requests can be lost, replies can be lost, slow calls time out after executing, the agent may re-issue an action after its retries fail, and crashed runs may replay the last call minutes later. A key store deduplicates repeats according to the key scope and retention you pick. Nothing here contacts a real service; all numbers are calculated estimates.
Play the lab
How to play
Choose the key scope, then adjust retries, client timeout, backoff and key retention. Every change replays the same 4,000 actions with the same random seed, so differences come from your change. The tiles show duplicate side effects per thousand actions, actions that never executed, p95 time until the agent has an answer, attempts per action (load on the tool service), the share of actions whose outcome stayed unknown, and how many keys the store must hold. Use New random seed to check a conclusion against different traffic.
Missions unlock in order. Load mission setup starts from a broken configuration. Events add a slow downstream, a lossy network and worker crashes with resume, so you can see which repeats each key scope catches.
What the model assumes
Actions arrive at 50 per second. Service latency is lognormal with a 300 ms median (four times slower with the slow event). One request in a hundred is lost before reaching the server; 2% of replies are lost (12% on the lossy network), and the client only notices at its timeout. A retry that finds the key still running gets a fast in-progress answer and backs off. When all retries end without an answer, the agent re-issues the action 60% of the time after a 1.5-second model step. With the crash event, 8% of runs replay the last call 2 to 20 minutes later. A per-call key changes on re-plans and replays; an intent key does not.
What it cannot tell you
The model ignores partial failures inside the tool, transactions that span several systems, leases on stuck in-progress records, and the possibility that a re-plan changes the arguments in a way that should count as a different operation. Its latency and loss figures are illustrative, not measured. Use it to build intuition for which repeats each design catches and how timeout and retention trade against latency and storage, then verify your own tools with fault-injection tests.
Read the full explanation
- Retrying Agent Tool Calls Without Executing the Same Action Twice (companion article)
- Why Agent Loops Run Away and How Step Budgets Stop Them
- Human Approval Gates That Keep Agents Safe Without Becoming the Bottleneck
All AI in Production labs · Lab catalog
- IETF draft: The Idempotency-Key HTTP Header Field
- RFC 9110: HTTP Semantics (idempotent methods)
- Amazon Builders' Library: Making retries safe with idempotent APIs
- Stripe API reference: Idempotent requests
- Model Context Protocol specification: Tools (tool annotations)
- LangGraph documentation: Durable execution
Views are my own and do not represent any employer, past or present.