Build evidence-backed findings and reviewed regression fixtures

Pattern 15 companion: Python example · Behavioral tests · Example license

Run the complete local example

Download pattern_15.py and test_pattern_15.py. With Python 3.11 or later, macOS commands are python3 -B pattern_15.py and python3 -B -m unittest -v test_pattern_15.py. Windows launcher equivalents are py -3 -B pattern_15.py and py -3 -B -m unittest -v test_pattern_15.py. The code uses only the standard library and has no network, model, or external mutation capability.

The output reports one finding, one unknown record, one duplicate, one reviewed fixture, and zero deployed actions. These counters describe a fixed synthetic sequence. They do not estimate a production failure rate or show that a model-based observer has been calibrated. Read the fixtures and assertions alongside the summary.

Architecture and request sequence

Component diagram for Build evidence-backed findings and reviewed regression fixtures — Original reference architecture by Sandeep Belgavi. Arrows show the responsibilities and transitions described below.

Request sequence for Build evidence-backed findings and reviewed regression fixtures — Illustrative sequence for this design; failure and recovery variants are explained in the article.

Inspect the immutable input records

Evidence and Session are frozen dataclasses. Evidence records the reference, source revision, structured outcome, completeness, and redaction status. Session records the session identifier, revision, claim, and evidence reference. The observer copies the supplied evidence mapping so later changes to the caller's dictionary do not replace entries inside the observer automatically.

The values still originate from a trusted synthetic caller. Frozen fields do not authenticate a producer or prove that a redaction flag is truthful. A production ingestion adapter needs provenance, validation, and access controls before constructing these records. The example deliberately keeps the claim as a small structured value rather than pretending that string matching can understand arbitrary incident conversations.

Follow inspection through identity and completeness

Inspection first checks the session-and-revision key. An exact duplicate returns duplicate; changed content under that key returns conflict. New records enter the seen map, then their evidence reference is resolved. Absent evidence, mismatched reference or revision, incomplete evidence, or an unsupported outcome produces unknown. Only supported complete evidence reaches the completion-conflict rule.

The rule creates a finding when the session claims completed but the evidence outcome is pending or failed. A completed outcome produces no_finding under this rule, not a claim that the whole incident is healthy. Tests separately exercise missing evidence, incomplete evidence, revision mismatch, a healthy case, and a genuine contradiction. These distinctions keep unavailable information from being silently counted as success.

Understand the reprocessing boundary

The seen map remembers unknown cases too. Repeating the same session in this observer instance therefore returns duplicate even if an operator has subsequently changed the evidence mapping. That is intentional for a bounded immutable batch. A production observer that supports evidence enrichment needs a new investigation generation or an explicit reprocessing operation, with separate audit history and deduplication semantics.

Do not mutate a prior result invisibly to make the dashboard look complete. Preserve which evidence version supported the original judgment and which supported the later judgment. A durable implementation can store investigation identity separately from session identity. The current module does not persist state across process restarts or coordinate multiple workers, and its tests make no claim about distributed ingestion.

Export only an approved, redacted finding

The promote_fixture method requires the approved flag, looks up an existing finding, and checks the evidence snapshot retained when the finding was created, including its redacted flag. It then stores the exact session and evidence pair in the fixtures map. The approval-denial and redaction-denial tests ensure that an unapproved or unsanitized finding cannot enter that map through this method. Another test verifies the exact exported pair.

This is an in-memory export boundary, not a file publisher. The flags are seams for review and sanitization services; a production system must bind approval to the evidence revision and verify the sanitization process. A future model adapter can propose a root-cause explanation, but it should not set its own approval flag or acquire deployment credentials. The example contains no pretend model response.

A replacement in the current lookup map cannot change the captured finding evidence. The regression test replaces that map entry between inspection and approval and verifies that export still contains the original immutable snapshot.

Connect storage, traces, and tests without overstating the result

A proposed service stores immutable evidence snapshots, findings, review decisions, and fixture versions separately. Use trace identifiers to locate relevant operations, following OpenTelemetry's correlation model, while keeping the business outcome record as independent evidence. The storage adapter needs transactional deduplication and retention rules; a dictionary is not a durable queue.

Eleven local tests cover classification, completeness, revision matching, duplication, conflicting identity, approval, redaction, and fixture contents. They do not establish production precision, root-cause accuracy, or cloud deployment. A reference deployment would run the observer with read access to approved evidence and write access only to its own findings and review stores. Actual credentials, scheduling, and model integration remain separate implementation work.

Keep the evidence producer separate from the fixture reviewer. Otherwise one component could label its own incomplete data trustworthy and approve the same unsupported conclusion for future acceptance tests.