HarmBench

HarmBench asks a different question from most safety evaluations: not whether a model refuses an obviously harmful request asked plainly, but whether it holds up once an attacker is actively trying to break it. Researchers from the University of Illinois Urbana-Champaign, the Center for AI Safety and several other institutions built a set of 510 harmful behaviours — cybercrime, chemical and biological weapons, misinformation, copyright infringement and more — and paired it with 18 automated jailbreak methods, from prompt optimisation to fine-tuning attacks. A model is scored on its attack success rate (ASR): the share of attempts, across methods, that produce a genuinely harmful response rather than a refusal.

The 2024 paper’s headline finding was that no single defence held up against every attack, and that robustness did not track model size — some small, deliberately hardened models resisted jailbreaks better than much larger ones with only ordinary safety tuning. The paper also introduced its own adversarial-training method, which cut one attack’s success rate on a 7-billion-parameter model from 31.8% to 5.9%, offered as a demonstration that HarmBench could drive defence development as well as measure it.

HarmBench does not run a live public leaderboard; instead, its open-sourced behaviours and attack suite have become a standard tool that outside researchers apply to individual models. That is how it produced one of its most-cited results: in early 2025, researchers at Cisco and the University of Pennsylvania used it to find that DeepSeek R1 refused none of 50 harmful prompts, a 100% attack success rate against a model then attracting heavy scrutiny, compared with 26% for OpenAI’s o1-preview, the most resistant model in their six-model comparison.