Why it matters

MMLU-Pro is modern knowledge benchmark. Understanding shapes eval.

Advertisement

The architecture

12k Qs across STEM + humanities + more.

10 choices per Q.

Chain-of-thought expected.

MMLU-Pro structure14 domainsSTEM + humanities10 choicesvs 4 in MMLUCoT-orientedharder reasoningBetter distinguishes frontier models than saturated MMLU; used widely
MMLU-Pro.
Advertisement

How it works end to end

TIGER-Lab curated.

Removed easy questions.

Added harder from other sources.

Zero-shot CoT scoring.