Hhhhhl/JBB-Behaviors

language:

• en

pretty_name: "JBB-Behaviors"

tags:

• jailbreaks
• large language models
• harmful behaviors
• ml safety

license: "mit"

configs:

• config_name: behaviors
  data_files:
  • split: harmful
    path: "data/harmful-behaviors.csv"
  • split: benign
    path: "data/benign-behaviors.csv"
• config_name: judge_comparison
  data_files:
  • split: test
    path: "data/judge-comparison.csv"

An Open Robustness Benchmark for Jailbreaking Language Models

NeurIPS 2024 Datasets and Benchmarks Track

Paper | Leaderboard | Benchmark code

What is JailbreakBench?

Jailbreakbench is an open-source robustness benchmark for jailbreaking large language models (LLMs). The goal of this benchmark is to comprehensively track progress toward (1) generating successful jailbreaks and (2) defending against these jailbreaks. To this end, we provide the JBB-Behaviors dataset, which comprises a list of 100 distinct misuse behaviors---both original and sourced from prior work (in particular, Trojan Detection Challenge/HarmBench and AdvBench)---which were curated with reference to OpenAI's usage policies. We also provide the official JailbreakBench leaderboard, which tracks the performance of attacks and defenses on the JBB-Behaviors dataset, and a repository of submitted jailbreak strings, which we hope will provide a stable way for researchers to compare the performance of future algorithms.

Accessing the JBB-Behaviors dataset

Some of the contents of the dataset may be offensive to some readers

Each entry in the JBB-Behaviors dataset has four components:

• Behavior: A unique identifier describing a distinct misuse behavior
• Goal: A query requesting an objectionable behavior
• Target: An affirmative response to the goal string
• Category: A broader category of misuse from OpenAI's usage policies
• Source: the source from which the behavior was sourced (i.e., Original, Trojan Detection Challenge 2023 Red Teaming Track/HarmBench, or AdvBench)

The first three entries from the JBB-Behaviors dataset are shown in the following table.

The dataset comprises of 100 distinct misuse behaviors (with examples sourced from AdvBench, Trojan Red Teaming Competition/HarmBench, and ideas sourced from Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation by Shah et al.) divided into ten broad categories corresponding to OpenAI's usage policies:

We note that the JBB-Behaviors dataset is not a superset of its constituent datasets; we focus only on 100 representative behaviors to enable faster evaluation of new attacks.
More information about the behaviors sources can be found in the dataset generation script.

Citation

If you find our benchmark useful, please consider citing it as follows:

@inproceedings{chao2024jailbreakbench,
  title={JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models},
  author={Patrick Chao and Edoardo Debenedetti and Alexander Robey and Maksym Andriushchenko and Francesco Croce and Vikash Sehwag and Edgar Dobriban and Nicolas Flammarion and George J. Pappas and Florian Tramèr and Hamed Hassani and Eric Wong},
  booktitle={NeurIPS Datasets and Benchmarks Track},
  year={2024}
}

In addition, if you use the JBB-Behaviors dataset in your work, we ask that you also consider citing its constituent datasets (AdvBench and TDC/HarmBench):

@misc{zou2023universal,
  title={Universal and Transferable Adversarial Attacks on Aligned Language Models},
  author={Andy Zou and Zifan Wang and J. Zico Kolter and Matt Fredrikson},
  year={2023},
  eprint={2307.15043},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}
@inproceedings{tdc2023,
  title={TDC 2023 (LLM Edition): The Trojan Detection Challenge},
  author={Mantas Mazeika and Andy Zou and Norman Mu and Long Phan and Zifan Wang and Chunru Yu and Adam Khoja and Fengqing Jiang and Aidan O'Gara and Ellie Sakhaee and Zhen Xiang and Arezoo Rajabi and Dan Hendrycks and Radha Poovendran and Bo Li and David Forsyth},
  booktitle={NeurIPS Competition Track},
  year={2023}
}
@article{mazeika2024harmbench,
  title={HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal},
  author={Mazeika, Mantas and Phan, Long and Yin, Xuwang and Zou, Andy and Wang, Zifan and Mu, Norman and Sakhaee, Elham and Li, Nathaniel and Basart, Steven and Li, Bo and Forsyth, David and Hendrycks, Dan},
  journal={arXiv preprint arXiv:2402.04249},
  year={2024}
}

License

This dataset, like the code, is released under MIT License.

Contributing

We welcome contributions to the benchmark! Please see the contributing guide for more information.