Start with a question that needs a plan
An incident assistant suspects checkout degradation but does not yet know which observations will help. A planner can select a small investigation plan: inspect a latency record, compare an availability record, and prepare findings. The executor follows accepted steps. A verifier assesses whether the resulting claims agree with a separately supplied evidence view. This pattern is useful when the investigation order is not entirely fixed, yet the application still needs bounded work and an explicit check before treating a finding as supported.
Our teaching design keeps all operations observational. A verified claim that a fixture is degraded does not authorize a restart, rollback or customer announcement. Those decisions belong to another operational policy. The architecture is therefore a chain of proposals and evidence checks, not three agents voting themselves permission to act. Its value comes from distinct responsibilities and interfaces, not from the number of model names in the diagram.
Architecture and request sequence
Read the architecture as text
- Planner seam: Proposes bounded read steps
- Plan validator: Rejects malformed work before execution
- Executor: Reads synthetic evidence mapping
- Separate verifier: Checks claims against another mapping
- Review result: Supported or unresolved, no remediation
- Planner seam → Plan validator: submit proposed steps
- Plan validator → Executor: execute accepted plan
- Executor → Separate verifier: supply report and references
- Separate verifier → Review result: publish narrow verdict
Read the sequence as text
- Caller → Planner: Request bounded investigation
- Planner → Executor: Propose read steps
- Executor → Verifier: Supply report and references
- Verifier → Executor: Check separately supplied evidence
- Verifier → Caller: Return narrow support result
Make the plan executable and rejectable
A plan is a finite list of steps. Each step has a stable key and an approved evidence reference. Validate the plan's size, element types, required fields and duplicate keys before executing any step. Rejecting a malformed plan before partial execution avoids an awkward state where the first observation ran but later elements turned out to be narrative text. In a production adaptation, also validate dependencies, service scope and estimated cost. The companion example intentionally supports only a flat list, not an arbitrary dependency graph.
The planner contract should state which questions it may ask and how missing information is represented. A reference outside the allowed inventory cannot become legitimate merely because the planner included it. The executor must enforce read scope independently. Store the accepted plan revision and planner revision so later evidence can be attached to the exact proposed work. A replanned investigation should preserve its predecessor rather than overwrite the original hypothesis.
Define independence at the evidence boundary
Using a different prompt for verification does not guarantee an independent check. If the verifier sees only the executor's polished summary, it can repeat the same error with different wording. Provide a separately retrieved or otherwise independently controlled evidence view and let the verifier compare the concrete claim with that view. In the fixture, execution reads one mapping and verification receives another. Tests make them disagree to demonstrate rejection. This is input separation, not proof of statistical independence between models or data sources.
Specify which properties the verifier actually checks. The small program checks that every reported claim is degraded and its referenced value is high in the independent mapping. It does not establish causality, timestamp alignment, authenticity or completeness of the incident investigation. A production verifier should report those dimensions separately. A single true flag should never imply more checks than the verifier implemented.
Budget three responsibilities without tripling waste
For an illustrative workload of twelve incident investigations per hour, a maximum three-step plan bounds executor reads at thirty-six before retries. Planning and verification add their own costs. If each step needs a large transcript, verifying every detail can cost more than planning the investigation. Decide which invariants are deterministic, which need expert labels and which warrant model assessment. Do not send the entire history to every role by default.
Bound replanning as carefully as initial planning. A disagreement can trigger one revised plan, human review or an explicit unresolved result. An unlimited planner-verifier conversation creates a hidden loop outside the original step ceiling. Record the number of plan revisions and the reason for each. Reserve enough budget to produce a useful unresolved report even when no plan passes verification; otherwise the system can spend all its allowance debating and leave the operator without evidence.
Know the tradeoff against simpler execution
Choose this pattern when planning uncertainty is real and verification can check something concrete. If the procedure is fixed, the preceding workflow pattern is easier to reason about. If the supposed verifier has no additional evidence and only judges writing style, the added component may increase latency without improving correctness. The decisive question is whether rejection corresponds to a meaningful operational difference that the team can reproduce and test.
Keep rejected findings available with their evidence and limitations. They may reveal stale telemetry, contradictory sources or an executor bug rather than a bad plan. Assign an owner to unresolved disagreements. Avoid designing the interface so every run must eventually receive approval; that pressure encourages weaker checks or repeated sampling until one result passes. A useful system can terminate with supported, unsupported or unresolved findings and explain why.
Sources and what this design establishes
The original plan and verifier source defines the narrow executable contract. Sandeep Belgavi's plan, workload assumptions and incident examples are educational designs. Consult Evals for measuring judges. No claim is made that independent mappings reproduce independent real-world evidence, or that passing the fixture verifier establishes production truth.