Detect agent stalls using evidence of progress

By Sandeep Belgavi · 18 October 2026

Combine task-specific progress signals with global turn, time and redirect limits.

Activity is not progress

A troubleshooting agent can produce many tool requests and lengthy explanations while repeatedly examining identical evidence. A turn counter measures length, not usefulness. Define task-specific progress: a newly verified artifact, fewer unresolved requirements, a previously failing test that now passes, or a confirmed operation. An agent saying that it made progress is not independent evidence that the task moved closer to completion.

Represent the accepted task state

For code repair, state might include the requirement revision, artifact digest, failing test identifiers and unresolved findings. Count a transition as progress only when new evidence meets a defined rule. Cosmetic changes should not reset a stalled bug-fixing workflow. Conversely, a diagnostic step may leave the artifact unchanged while eliminating a hypothesis. The rule should recognize reviewed diagnostic evidence without accepting arbitrary conversational claims.

Combine local and global limits

The exercise detects two unchanged observations. A real harness also needs maximum elapsed time, total turns and specialist redirects. Otherwise it can alternate between agents or slightly different states forever. Every redirect consumes the same shared budget as ordinary work. Preserve previous attempts so a new specialist does not repeat them without justification. Avoid resetting global counters when a local agent object is reconstructed after a failure.

Make escalation actionable

When execution stops, report the latest verified state, attempted actions, observed errors, unresolved questions and the constraint that ended the run. Ask for the smallest decision needed to proceed. A vague request to try again wastes operator attention. A new run requires an explicit budget and current permissions. A prior worker request for help must not grant unlimited continuation or preserve access that has since been revoked.

Test false stalls and false progress

Include a slow useful investigation, repeated failed tool calls, a completed operation with a lost response and cosmetic state changes. Evaluate premature stopping alongside wasted computation. Reducing average turns by abandoning hard tasks is not an improvement. Compare completed outcomes and escalation usefulness by difficulty. The deterministic fingerprint rule below is a teaching device; its threshold is not evidence of production suitability.

Keep recovery evidence outside summaries

A model-written summary can describe progress but should not become the only checkpoint. Store accepted artifacts, validation outcomes and pending effects in durable application records. On restart, reconstruct the next safe step from those records, then use a summary as context. This separates continuity from authority. A corrupted or overly optimistic summary should not make the harness skip validation, repeat a completed effect or declare an unresolved task finished.

Run the exercise

Download this Python exercise

Save the code as 11-stalls-and-progress-signals.py and run python 11-stalls-and-progress-signals.py with Python 3.11 or newer. This standard-library exercise uses synthetic data. It does not invoke a model or require an account, network connection or API key.

# Author: Sandeep Belgavi. Synthetic exercise; no network or provider calls.
fingerprints=["requirements-v1","artifact-v1","artifact-v1","artifact-v1"]
previous=None
unchanged=0
status="RUNNING"
for fingerprint in fingerprints:
    unchanged=unchanged+1 if fingerprint==previous else 0
    previous=fingerprint
    if unchanged>=2:
        status="ESCALATE"
        break
assert status=="ESCALATE"
print("escalate after two unchanged observations")

Expected output:

escalate after two unchanged observations

Change one assumption

Give every observation a new fingerprint and notice that the detector never escalates. Add an independent four-turn ceiling. Explain why trusted artifact validation must determine whether a new fingerprint counts as progress. Test a legitimate diagnostic step separately so the stronger policy does not reject useful investigation merely because a source file stayed unchanged.

Implementation scope and further reading

This is an original AICassindra teaching design. The exercise demonstrates the stated invariant; it is not a complete production implementation. The local Harness Studio extension supports saved read-only agent teams, reviewed count-based routing and bounded sequential execution. Arbitrary imported graphs, effectful tools and automatic self-promotion remain unsupported.

All harness articles · Interactive boundary labs · Our public lab code