Agent Tool Effects Through a Transactional Outbox: One Transaction for the Step Journal and the Side Effect, an At-Least-Once Relay and Resume After a Crash

The dual write hides inside every tool call

A service that updates a row and then publishes an event has a dual-write problem: if the process dies between the two writes, the database says one thing and the message broker another. The transactional outbox fixes it by writing the event into an outbox table in the same transaction as the business change, and letting a separate relay publish it later. The classic version is covered in the outbox pattern with a polling relay or CDC.

An agent tool coordinates three records: the business change, the execution journal and downstream delivery intent. Writing them separately creates crash windows. A lost response can trigger a retry even after a refund committed, so storage must distinguish completed work from work still pending.

What stays deterministic

Anything that can be stated over structured data is enforced by the database and the relay; the model keeps the decisions that need judgement. The runtime accepts identical retries and rejects changed intent; remote delivery still needs consumer deduplication.

Three rows in one transaction

Give the runtime one relational database for its own state and route every effecting tool through it. The handler opens a transaction, applies the business change, inserts the journal row keyed by run id and step index, inserts the outbox row, and commits. Die before the commit and none of the rows exists, so resume tries the step again. Die after it and all three exist; the agent may never see the response, and the step key makes the retry return the journaled result instead of executing twice.

Illustrative PostgreSQL schema; the refunds table belongs to the business domain.

CREATE TABLE step_journal (
  run_id      text NOT NULL,
  step_index  int  NOT NULL,
  tool        text NOT NULL,
  request_hash text NOT NULL,
  result      jsonb,
  committed_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (run_id, step_index)
);
CREATE TABLE outbox (
  id          bigserial PRIMARY KEY,
  run_id      text NOT NULL,
  step_index  int  NOT NULL,
  topic       text NOT NULL,
  payload     jsonb NOT NULL,
  published_at timestamptz
);

Illustrative python, checked for syntax; adapt names and storage to your system.

import hashlib
import json

def issue_refund(db, run_id, step_index, order_id, amount):
    """Effect, journal and outbox commit together or not at all."""
    if type(amount) is not int or amount <= 0:
        raise ValueError("amount must be positive integer minor units")
    intent = json.dumps(["issue_refund", order_id, amount], separators=(",", ":"))
    request_hash = hashlib.sha256(intent.encode("utf-8")).hexdigest()
    with db.transaction() as tx:
        done = tx.one("SELECT tool, request_hash, result FROM step_journal WHERE run_id=%s AND step_index=%s",
                      run_id, step_index)
        if done:
            if done["tool"] != "issue_refund" or done["request_hash"] != request_hash:
                raise ValueError("step key reused with different intent")
            return done["result"]
        refund_id = tx.one("INSERT INTO refunds(order_id, amount) VALUES (%s,%s) RETURNING id",
                           order_id, amount)["id"]
        result = {"refund_id": refund_id, "amount": amount}
        tx.run("INSERT INTO step_journal(run_id, step_index, tool, request_hash, result) VALUES (%s,%s,%s,%s,%s)",
               run_id, step_index, "issue_refund", request_hash, result)
        tx.run("INSERT INTO outbox(run_id, step_index, topic, payload) VALUES (%s,%s,%s,%s)",
               run_id, step_index, "refund.issued", result)
    return result

The stable step key identifies the operation; the tool and canonical semantic-argument hash reject conflicting reuse. Amounts are positive integer minor units. Concurrent first attempts can race: the journal primary-key conflict must roll back the losing transaction, which then reloads and validates the winner. Only local rows and delivery intent commit together; remote effects need consumer deduplication. See the idempotency guide.

The relay and the consumer

The relay is a loop that selects unpublished outbox rows, publishes them, and marks them published. It can run in several replicas if the select locks rows and skips locked ones, which PostgreSQL offers as FOR UPDATE SKIP LOCKED. The order of the two relay actions decides the delivery semantics. Publish first and then mark: a crash between the two republishes the row, so delivery is at least once. Mark first and then publish: a crash between the two loses the message. At least once is the right choice, which means the consumer must deduplicate on the outbox id, a single integer that arrives inside every message.

Illustrative python, checked for syntax; adapt names and storage to your system.

def relay_once(db, broker, batch=100):
    with db.transaction() as tx:
        rows = tx.all("""SELECT id, topic, payload FROM outbox
                         WHERE published_at IS NULL
                         ORDER BY id LIMIT %s FOR UPDATE SKIP LOCKED""", batch)
        for row in rows:
            broker.publish(row["topic"], dict(row["payload"], outbox_id=row["id"]))
        ids = [row["id"] for row in rows]
        if ids:
            tx.run("UPDATE outbox SET published_at = now() WHERE id = ANY(%s)", ids)
    return len(ids)

def consume(db, message):
    with db.transaction() as tx:
        inserted = tx.run("INSERT INTO processed(outbox_id) VALUES (%s) ON CONFLICT DO NOTHING",
                          message["outbox_id"])
        if inserted == 0:
            return "duplicate"                    # already applied by an earlier delivery
        tx.run("INSERT INTO ledger_entries(refund_id, amount) VALUES (%s,%s)",
               message["refund_id"], message["amount"])
    return "applied"

With change data capture instead of polling, the consumer contract is unchanged; a runnable model with tests injects each crash.

Worked example: a refund issued by an agent

A support agent decides on a refund of 40 for order 7731 and calls issue_refund as step 2 of run R-91. Walk the crash points. The handler dies after inserting the refund but before commit: the transaction rolls back, resume finds step 2 pending and runs it again, one refund. The handler commits and dies before answering: the agent retries step 2, the journal lookup returns the stored result, one refund. The relay publishes and dies before marking: the next pass publishes again, the consumer sees the same outbox id and applies nothing. The whole host is replaced mid-run: the new host loads R-91 from the journal, sees steps 1 and 2 committed, and asks the model for step 3 with the journaled results in context.

Now remove one safeguard at a time. Without the single transaction, the first crash point leaves a refund with no journal row and no event, and the retry issues a second refund nobody hears about. Without the step key, the second crash point produces two refunds even with the transaction. Without consumer deduplication, the third posts the refund to the ledger twice. The lab below lets you move the crash point and switch safeguards off.

Agent retries are the normal case

A model-driven loop retries more often than a hand-written client: a timeout becomes an error string and the model asks again; a reflection step calls the tool once more with the same arguments. These are not bugs to prompt away; they are why the step key exists. The runtime must therefore assign step indices before the call and keep them stable across a retry, so the journal lookup hits; a retry treated as a new step defeats the scheme, which happens when the step counter lives in the prompt rather than the runtime.

Read-only tools need none of this. A lookup can run twice without harm; journaling its result helps resume but can happen after the fact. Classify tools by whether they have effects and route only the effecting ones through the transactional handler, the same read-or-write split the parallel tool call guide uses for scheduling.

Failure modes worth testing

• Two stores. The journal lives in the agent framework, the effect in the business database. There is no transaction across them, so every crash point is a dual write. Test by killing the handler after the business write.
• Step key from arguments. Hashing the arguments makes a retry with reworded text a new step. Derive the key from run id and step index.
• Relay marks before publishing. Delivery becomes at most once. Test by crashing the relay between its two statements.
• Consumer deduplicates in memory. A restarted consumer forgets what it processed. The processed table must be durable and written with the consumer effect.
• Resume without the journal. A resume loop that asks the model for the next step before loading the journal replays completed steps.

Migration steps for an existing agent runtime

1. Give the runtime a relational journal keyed by (run id, step index); record every tool result there, after the fact at first. Classify tools into effecting and read-only.
2. For effecting tools in the same database, move the journal insert into the tool transaction; for external systems, pass the step key as the external idempotency key and journal the response.
3. Add the outbox table and the relay, and replace every direct publish from a tool handler with an outbox insert in the same transaction.
4. Add the processed table to each consumer; make the runtime assign step indices before the call and load the journal before asking the model anything on resume.
5. Inject faults after the business write, after the commit and between the relay actions; each should leave exactly one effect and one applied event.

When an outbox and an agent are unnecessary

If the tool has no side effects, skip all of this. If the agent calls one effecting tool whose owner already offers idempotency keys and emits its own events, the step key is enough. The outbox earns its place when the runtime owns the effect and other systems must learn of it reliably. And if the sequence of tool calls is fixed, there is no agent in the picture: a plain workflow with the same journal, outbox and relay is cheaper and fully testable.

Practice and related reading

Hands-on: Agent outbox relay: crash points, duplicates and resume, an interactive lab on synthetic data.

Further reading:

• Transactional outbox: polling relay versus Debezium CDC
• Transactional outbox pattern: solving the dual-write problem
• adk-java transactional outbox architecture
• Agent checkpointing, in depth: step boundaries, write ordering and idempotent resume
• Agent-proposed plans inside an existing saga orchestrator
• Idempotency in system design

Sources and scope

Checked against these sources on 19 October 2026. Examples and figures are illustrative, not measurements.

• microservices.io: Transactional outbox pattern
• PostgreSQL documentation: SELECT, the locking clause (FOR UPDATE, SKIP LOCKED)
• Stripe API reference: idempotent requests
• Restate documentation: durable execution and the journal