AdvBench — Adversarial Benchmark

Why it matters

AdvBench is jailbreak eval baseline. Understanding shapes attack eval.

The architecture

~500 harmful behaviors + strings.

Test if model responds affirmatively.

Attack success rate metric.

How it works end to end

Zou et al. 2023.

Automated evaluation via string matching.

Complement with judge for nuance.