Design a fixed workflow around uncertain model outputs

Pattern 02 companion: Python example · Behavioral tests · Example license

Make the procedure the stable part

An incident desk accepts a new report, classifies the symptom, collects the corresponding evidence and prepares a review package. The sequence is known before any model sees the report. This is a good candidate for a deterministic workflow: code owns progression, while a model may supply a category at the classification step. The model cannot decide that evidence collection is unnecessary or jump directly to a remediation command. The architecture makes procedural consistency independent of how fluent a generated explanation sounds.

Our original example supports latency and availability incidents. These labels deliberately describe evidence needs rather than root causes. A report classified as latency still requires measurements; classification alone does not establish that a database, network or recent deployment caused the symptom. Keep that distinction visible in the state contract. A workflow that treats a category as a confirmed diagnosis can follow every transition correctly and still produce an unjustified operational recommendation.

Architecture and request sequence

Component diagram for Design a fixed workflow around uncertain model outputs — Original reference architecture by Sandeep Belgavi. Arrows show the responsibilities and transitions described below.

Request sequence for Design a fixed workflow around uncertain model outputs — Illustrative sequence for this design; failure and recovery variants are explained in the article.

Specify stages as durable facts

Define new, classified, collected and ready as separate states. New means no accepted category exists. Classified means the category passed the supported-value check. Collected means the required evidence locator has been obtained. Ready means a package is available for review, not that the incident is resolved. A checkpoint carries the incident identity, current stage, accepted category and evidence locator. The transition function accepts one checkpoint and returns its successor; it does not mutate the caller's object.

For a service deployment, attach a schema version, workflow revision and source-event identifier to that checkpoint. The current teaching dataclass omits those fields to keep the state machine readable. Introduce them before storing checkpoints across releases. If a future version adds a security classification stage, an old classified checkpoint cannot safely be interpreted as having passed it. Migrate explicitly or route the old run to a compatible worker.

Put uncertainty inside typed slots

The classifier seam accepts incident text and proposes one category. A production adapter should return a constrained value plus supporting references, while the workflow validates the category against its allowlist. Unknown or ambiguous inputs need an explicit escalation path. Do not silently map everything unfamiliar to latency merely to keep the pipeline moving. The teaching implementation raises a validation error for unsupported values, leaving the original checkpoint unchanged so the caller can decide how to record and handle the failure.

Model judgment may also help summarize collected evidence, but that would be a different typed step with its own acceptance rule. Do not hide a second reasoning operation inside the collector. The collector's contract should remain observable: given an approved category and incident scope, return the required evidence or report that it is missing. Keeping these contracts small helps distinguish uncertain classification from failed ingestion.

Estimate capacity by stage

Suppose an illustrative intake receives twenty reports per minute. Only the new stage calls the classifier; retries after evidence collection should not call it again. If classification averages two seconds, the load on that stage differs from a collector that waits ten seconds for a telemetry window. A single end-to-end concurrency setting can conceal this imbalance. Separate queue depth and occupied workers by stage, then place backpressure before the scarce dependency.

The workflow's predictability helps capacity planning because the successful path has a known number of transitions. It does not eliminate uncertainty in stage duration or retry volume. Budget classifier requests, collector queries and stored payload size independently. For a burst, it may be better to delay new classifications while completing already classified cases. Make that scheduling decision explicit rather than letting whichever asynchronous worker wins a race determine operational priority.

Choose this pattern for auditability, not rigidity

A fixed workflow works well when policy requires particular checks and the branching space is small. It makes missing steps visible and supports focused rollback of a classifier adapter. Its cost is reduced flexibility: a novel incident may need a human or another pattern rather than an improvised sequence. Add a bounded exception path instead of gradually turning every transition into a free-form model decision. Otherwise the system retains the appearance of a workflow while losing its procedural guarantees.

Model-driven stages can evolve without changing the top-level procedure, provided their input and output contracts remain compatible. Version the rubric and adapter separately from the workflow definition. Compare classification quality on a labeled set before promotion, then monitor how often operators override the result. A lower exception count is not automatically improvement; it can mean the new classifier is forcing uncertain cases into familiar labels.

Sources and design boundaries

The companion transition implementation is the primary source for this original teaching workflow by Sandeep Belgavi. Classification quality should be measured using the concepts in Evals. The capacity figures are illustrative assumptions. This design does not claim a vendor workflow service, distributed exactly-once execution, a real model integration or an operational deployment.