Benchmarks · Safety, security & robustness
JailbreakBench
How well a jailbreak attack defeats a model's safety training, and how well a defence holds up against a standard set of attacks — tracked as an open, ongoing leaderboard rather than a one-off score.
University of Pennsylvania, EPFL & collaboratorsReleased 28 March 2024Live
Jailbreak research has a reproducibility problem: two papers claiming a new attack “breaks” a model often used different target behaviours, different judges of what counts as a successful jailbreak, and different, undisclosed prompting setups, making results hard to compare. JailbreakBench, built by researchers at the University of Pennsylvania, EPFL and collaborating institutions, was designed to fix that by standardising all three: a fixed set of 100 harmful behaviours (with 100 matched benign behaviours as a control), a shared repository of submitted attack prompts, and a single scoring pipeline that other work can cite directly.
Rather than publishing one headline number, JailbreakBench runs as a continuously updated leaderboard with separate tables for open-source and closed-source target models, so an attack or a defence can be checked against the same fixed baseline as everything that came before it. The behaviours themselves were drawn from OpenAI’s usage policy categories and cross-referenced against earlier jailbreak datasets, including HarmBench and AdvBench, so results are at least loosely comparable across the wider jailbreak-testing literature.
The paper was accepted to NeurIPS 2024’s Datasets and Benchmarks track, and the project has continued to accept new attack and defence submissions since. Because it is structured as a live leaderboard rather than a fixed test with a single winner, JailbreakBench functions less like a capability benchmark and more like a shared instrument that safety researchers calibrate their claims against.
The set
100 harmful behaviours plus 100 matched benign ones, drawn from OpenAI's usage policy categories and from earlier sets including AdvBench and HarmBench, with a public repository of submitted jailbreak prompts and a standardised scoring pipeline; separate leaderboards track open-source and closed-source target models.
Example
The 100 harmful behaviours span ten categories mirroring OpenAI's usage policy — including malware/hacking, fraud, disinformation, harassment and physical harm — ten per category, each paired with a matched benign behaviour testing the same skill without the harmful intent.
Where it stands
Accepted to NeurIPS 2024's Datasets and Benchmarks track; the project maintains a standing, updated leaderboard rather than reporting a single top score, so this file does not list a leader.