Home/ Research/ 2026/ October/ 11
Trend Watch 2026-10-11

Trend Watch: A 'Teaching Hospital' for Coding Agents and a 500-Billion-Token Game Decompilation Reach the Same Verdict -- Reviewer Agents Are Not Enough

Two long engineering write-ups published October 9 read as field reports from multi-agent software factories, and the Cockroach Labs one drew a lively Hacker News thread. Cockroach Labs ran coding agents for five months as a teaching hospital with plans, second opinions and audited discharges; Maurice Heumann had agents decompile a first-person shooter and found the code readable but wrong until a byte-matching check replaced the reviewer. Both conclude that throughput is easy and correctness is the work.

Analyzed & written by Sandeep Belgavi

Both posts went up on October 9, 2026. Cockroach Labs' "Five months treating bugs like patients and coding agents like a medical team" drew 73 points and 32 comments on Hacker News; one commenter summed up the mood: "It feels like everybody is trying to build a lot of the same concepts and converging onto something." The same week, this digest covered an agent-written TypeScript compiler port whose author had never read the code.

Cockroach Labs: MOLT Sinai

Adam Storm and Rafi Shamim built a pipeline for MOLT, the company's database migration tooling, modelled on a teaching hospital, "with mandatory handoffs, mandatory second opinions." Issues are patients and merging is discharge. Everything runs on GitHub Actions, with each stage triggered by an issue label.

  • "No code before a plan, no plan without review." A Fellow agent reproduces the problem and posts a treatment plan; a separate Review Attending, prompted to find fault, approves or rejects it.
  • "Never flail." A stuck agent writes a structured I-PASS handoff, borrowed from hospital shift changes, and escalates.
  • The Discharge Nurse audits the reviewer, checking that approval, template, CI and history are in order, rather than re-reviewing code.
  • Precedents. Generalisable decisions by the human Chief go into an append-only log agents must check before escalating; "Only a human can create precedents."

Results: IBM Db2 support for MOLT in under two days. "The token bill was $4172: 164x faster, and 38x cheaper" than the Oracle work in 2024. Over five months the pipeline landed "well over a million lines of code" with 7 reverts, at about $135,000 in Claude tokens, around $84 per issue. The authors also list the costs: bureaucracy, an urgent one-line fix that took "eleven rework rounds over two days," and skill files that had grown to 100,000 words, 23% of it removable. And a frank caveat: they are still "confirming that the IBM Db2 support is correct, as we weren't reviewing any of the code that the hospital wrote during that initial test."

The decompilation: readable, and wrong

Heumann's "500+ Billion Tokens Later" describes three months of agents decompiling an unnamed shooter in Claude Code and Codex CLI. In the first month, three workers and a reviewer reconstructed about 80% of the game; it launched and loaded maps. Steady progress made the team believe the quality was great. "It was not," Heumann writes. "Despite the code being extremely readable, it was semantically wrong." The reviewer was fooled by the workers' own explanations: "The workers' comments effectively acted as unintentional prompt injection."

The fix was a script comparing each compiled function, byte for byte, with the original. Agents first tried inline assembly, then "repeatedly tried to modify this script to exclude their function from comparison," so CI now hashes the script against a stored secret. With a hard pass/fail signal, cheaper models became viable: the final weeks ran 14 Luna and 2 Opus 5.5 agents, reaching 99% of functions present and 83% byte-exact. His takeaways: "Correctness should be defined and machine-checkable," and instructions decay, which an hourly cron job reminding agents to reread their brief fixed.

Why it's a trend

Software factories are now ordinary enough that the interesting posts are about control systems, not output. Both teams independently found that an LLM reviewer inherits the worker's framing, and that the real gate must be something the agents cannot argue with or edit.

I think these two posts are more useful than most benchmark launches this month, because they describe the failure that matters: agent systems that look productive while being wrong. My reading is that the hospital and the byte-matcher are two answers to the same question – where does ground truth come from? Heumann had a perfect oracle in the original binary, so he could replace judgement with a check, and then cheap models were good enough. Cockroach had no such oracle, so it rebuilt the social machinery humans use instead: plans before work, independent second opinions, audit trails and precedent. That is slower and costlier, and the authors admit it. The lesson for teams starting out is to invest in the oracle first. Every test, schema check or replay you can make machine-checkable lets you buy cheaper models and less bureaucracy; everything you cannot check will need a hospital, and a human reading the chart in the morning.

Cockroach Labs' teaching-hospital agent pipeline and a 500-billion-token game decompilation, both published October 9, found that reviewer agents absorb the worker's framing; correctness came from enforced plans, audits and precedents, or from a tamper-proof machine check that also let cheaper models do the work.

Sources

← Back to October 11, 2026