Why it matters

Autonomy evals shape agent risk. Understanding shapes safety.

Advertisement

The architecture

Long-horizon tasks with tools.

Sandbox with escape scenarios.

Success rate by task.

Autonomy eval structureLong-horizon tasksSWE + browsing + etc.Sandbox evalwith tools + isolationSuccess rateas capability growsFrontier lab RSP/PF include autonomy thresholds; METR + Apollo evaluate
Autonomy evals.
Advertisement

How it works end to end

METR autonomous replication evals.

SWE-bench + GAIA scale.

Time-scale of tasks measured.