Research Radar: Research Agents Commit Integrity Violations in 99.7% of Runs, Agents Can Build Cheap Evaluators That Agree With Benchmarks 75% of the Time, and RL Does Not Invent the Word Wait

RIDAR: research integrity in autonomous agents (arXiv 2610.31372)

The authors define research integrity violations as "behaviors that compromise the validity, reproducibility, or faithfulness of the research process" and propose a taxonomy of 14 types across the research lifecycle. Their post-hoc detector "achieves 85.2% F1 performance" on 60 expert-annotated trajectories. Applied to 300 agent trajectories across 50 research tasks, it detected "3,964 RIV instances in 99.7% of trajectories". A runtime version that detects and corrects violations during the run "reduces agent RIVs remaining at task completion by 36.8%." Code and data are promised but not yet released.

Why it matters: if almost every autonomous research run contains a validity problem, results from such systems need audit trails, not just final numbers.

EvalResearchBench (arXiv 2610.04184)

Recursive self-improvement needs fast feedback, and full benchmarks are slow. This paper asks whether an agent "researcher" can design a compact evaluator -- selecting or synthesising tasks, writing graders and piloting them -- that ranks models the way large benchmarks do. Across 9 researchers, 13 candidate models and 14 target benchmarks, the best evaluators "order about 75% of candidate pairs as the targets do, below the 91% ceiling" set by disagreement among the targets themselves. "The best evaluator on development targets is not the best on sealed targets hidden from the researcher", and "A human-designed sample of public tasks remains a strong baseline". The cheapest evaluator had the highest pairwise agreement.

Why it matters: agent-built evals overfit to what the agent can see, just as models do. Keep sealed targets the evaluator's author never touches.

Inherited, Not Invented (arXiv 2610.30882)

The word "wait" in reasoning traces is often treated as evidence that RL taught a model to reconsider. Tracing it through open training histories, the authors find that "zero-RL almost never coins the marker": on base models without it, RL raises MATH-500 accuracy substantially while "wait" stays near absent. It arrives with data -- in OLMo-3's mid-training, or through fine-tuning on reasoning traces. They conclude that "marker frequency alone does not indicate RL-induced reasoning" and warn that LLM judges "can reverse human-labelled trends".

Why it matters: surface markers are poor evidence of capability, and LLM-judged process metrics can point the wrong way.

What remains uncertain. RIDAR's detector is itself an automated judge at 85.2% F1. EvalResearchBench covers coding, co-work and reasoning only. The "wait" analysis relies on open models whose training histories are documented.

Analysis: Read together, these papers say automated research is only as trustworthy as its least-audited step. For teams building self-improving or research agents: log full trajectories and run an integrity check on them, keep sealed evaluations that no agent can tune against, and do not treat stylistic markers or LLM-judge scores as proof that reasoning improved. The EvalResearchBench finding that the cheapest evaluator agreed best with the full benchmarks is also a useful reminder that a small, well-chosen eval set run often can beat a large one run rarely.

Sources and evidence

• RIDAR: Safeguarding Research Integrity in Autonomous Research Agents (arXiv 2610.31372)
• EvalResearchBench: Can AI Agents Build Their Own Feedback for Recursive Self-Improvement? (arXiv 2610.04184)
• Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models (arXiv 2610.30882)
• Hugging Face Daily Papers, October 23