A restart should not invent a new incident
An incident assistant has read a queue diagnostic and prepared a summary when its process stops. On restart it should know whether it needs to read again, whether a summary already exists and whether any action was committed. A transcript alone cannot answer those questions reliably. It may contain an intended action, a tool timeout or a confident statement of completion without an authoritative receipt.
Our proposed workflow has three explicit states: not inspected, evidence recorded and local report completed. The state is deliberately small. It makes the recovery question answerable without asking a model to reconstruct progress from prose. A production workflow can have more states, but every state should explain what is durable, what is safe to repeat and which operation owns the next transition. Durability is a contract about those transitions, not a promise that failures disappear.
Architecture and request sequence
Read the architecture as text
- Coordinator: Versioned state machine
- Read adapter: Uncertain observation
- Checkpoint store: Step and evidence together
- Receipt table: Local completion only
- Recovery worker: Resume known state
- Coordinator → Read adapter: inspect
- Read adapter → Checkpoint store: validated evidence
- Coordinator → Receipt table: local completion
- Checkpoint store → Recovery worker: reopen
- Recovery worker → Coordinator: resume step
Read the sequence as text
- Worker → Store: Load version and step
- Worker → Provider: Read incident evidence
- Worker → Store: Commit checkpoint
- Worker → Store: Commit local receipt
- Recovery → Store: Resume recorded state
Separate the workflow from uncertain work
The coordinator decides which step comes next. A provider adapter performs the uncertain observation, and the database records the result. The reference adapter returns a synthetic queue diagnosis; a real adapter could read telemetry or call a model. Its response must satisfy the evidence contract before a checkpoint advances. A provider response is data, not permission to rewrite the coordinator's state machine.
Temporal documents replay against recorded event history as part of resuming workflow executions. Our SQLite example uses explicit state rows rather than implementing Temporal replay. The shared lesson is to identify which results belong in durable history. Do not claim that a small checkpoint table reproduces a distributed workflow service, its scheduling behavior or its operational guarantees.
The checkpoint is not the remote effect
The first transition stores evidence and advances the step in one local transaction. The second writes a local receipt and marks completion in another transaction. Because the receipt and step share the same database, the reference can roll them back together. That useful property does not extend to a remote service restart, ticket creation or payment made through a network API.
For an external effect, record a stable operation identity and reconcile ambiguous outcomes with the destination. A timeout after sending is not evidence that nothing happened. If the destination cannot provide idempotency or a queryable receipt, the workflow needs an explicit uncertain state and a recovery procedure. Retrying blindly can duplicate harm; marking complete without evidence can lose work. Neither problem is solved by saving more chat history.
Version every long-lived decision
A workflow may be paused when a new application release changes the meaning of a step. The reference stores a workflow version with every run and refuses to interpret a row under another version. This is intentionally conservative. It prevents a new implementation from silently treating yesterday's step one as today's entirely different operation, but it also means upgrades need an explicit migration or compatibility strategy.
Store provider model, prompt, tool schema and evidence revisions separately in a real system. The workflow version describes control flow; those other versions describe the decision inputs. A completed observation should not be recomputed merely because a newer model is available. Re-evaluation is a new, identifiable operation with a reason and comparison output. Otherwise replay can quietly alter a past decision while appearing to resume the same run.
Capacity and state retention
Assume a trial workload of one hundred incidents per day, three checkpoints per incident and evidence references averaging two kilobytes. That is small enough for many storage choices, but it says nothing about provider latency or burst concurrency. Retaining raw telemetry inline can dominate storage and expose sensitive data. Prefer bounded evidence records plus governed references where the underlying retention policy permits reproducible investigation.
The lab calls its adapter while a SQLite write transaction is open. This simplifies the demonstration and is unsuitable for slow remote calls at scale because it extends lock duration. A production design should lease work, perform the call outside the transaction and conditionally commit against the expected state version. That change introduces lease expiry and stale-worker races, so add fencing and concurrency tests rather than presenting the optimization as a free improvement.
Choose a small, testable guarantee
The original module promises that reopening the same database retains a completed evidence checkpoint, a failed local transaction does not advance the state, and repeated completion leaves one local receipt. It does not promise that an uncommitted provider call happens only once. If a crash occurs after reading but before committing, the read can happen again. That behavior is visible in a dedicated test.
Use these distinctions to choose an engine and deployment topology. An embedded state machine can be reasonable for a bounded local worker; distributed timers, long waits, many concurrent runs and controlled upgrades may justify a dedicated workflow platform. In either case, evaluate the same business outcomes through the evaluation layer. A recovered workflow can still produce a wrong diagnosis, so recovery correctness and task correctness must remain separate acceptance criteria.