Why it matters

MMLU-style safety tests dual-use knowledge. Understanding shapes capability eval.

Advertisement

The architecture

WMDP: weapons of mass destruction proxy.

MMLU subject subsets.

Multiple choice for automated scoring.

MMLU safety subsetsWMDPweapons proxyMMLU subsetsbiosec / cybersec / chemMC scoringautomatedLi et al. 2024 WMDP; benchmarks dual-use knowledge without direct harm
MMLU safety.
Advertisement

How it works end to end

WMDP: bio + chem + cyber.

Multiple choice enables automation.

Measure knowledge, not action.