license: cc-by-nc-4.0
task_categories:
- text-generation
language: - en
tags: - adversarial robustness
- human red teaming
extra_gated_fields:
Name: text
Affiliation: text
Email (affiliation email if possible): text
Country: text
Purpose: text
size_categories: - n<1K
Dataset Card for MHJ (08/27/2024)
This card contains datasets for the paper LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks.
We compile successful jailbreaks into the Multi-Turn Human Jailbreaks
(MHJ) dataset, consisting of 2,912 prompts across 537 multi-turn conversations. We include
relevant metadata for each submission, including design choice comments from each red teamer for their
jailbreak. The resulting attack success rate (ASR) of our human red teaming is shown as follows.
*CYGNET is closed
source, hence results for AutoDAN, GCG, and PAIR are cited from the original papers and should
not be directly compared against human ASR.
Due to potential export control concerns, we omitted some of the HarmBench behaviors and redacted some of the completions. We examined the HarmBench test set, standard and contextual behaviors for this research. For the same reason, we omitted the responses for WMDP-bio behaviors (if you just want to download the same WMDP-bio set we are using, please find here: https://huggingface.co/datasets/ScaleAI/mhj-wmdp-bio.
Paper Abstract
Recent large language model (LLM) defenses have greatly improved models’ ability to refuse harmful
queries, even when adversarially attacked. However, LLM defenses are primarily evaluated against
automated adversarial attacks in a single turn of conversation, an insufficient threat model for real-
world malicious use. We demonstrate that multi-turn human jailbreaks uncover significant vulnerabilities,
exceeding 70% attack success rate (ASR) on HarmBench against defenses that report single-digit ASRs
with automated single-turn attacks. Human jailbreaks also reveal vulnerabilities in machine unlearning
defenses, successfully recovering dual-use biosecurity knowledge from unlearned models. We compile
these results into Multi-Turn Human Jailbreaks (MHJ), a dataset of 2,912 prompts across 537 multi-turn
jailbreaks. We publicly release MHJ alongside a compendium of jailbreak tactics developed across dozens
of commercial red teaming engagements, supporting research towards stronger LLM defenses.
Bibtex Citation
If you use this dataest, consider to cite
@misc{li2024llmdefensesrobustmultiturn,
title={LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet},
author={Nathaniel Li and Ziwen Han and Ian Steneker and Willow Primack and Riley Goodside and Hugh Zhang and Zifan Wang and Cristina Menghini and Summer Yue},
year={2024},
eprint={2408.15221},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2408.15221},
}