language:

• en
  dataset_info:
  features:
  • name: prompt
    dtype: string
  • name: platform
    dtype: string
  • name: source
    dtype: string
  • name: jailbreak
    dtype: bool
    splits:
  • name: train
    num_bytes: 27480306
    num_examples: 15140
    download_size: 14209695
    dataset_size: 27480306
    configs:
• config_name: default
  data_files:
  • split: train
    path: data/train-*
    license: mit

In-The-Wild Jailbreak Prompts on LLMs

Paper: [``Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://arxiv.org/pdf/2308.03825)

Data: Dataset

Data

Prompts

Overall, authors collect 15,140 prompts from four platforms (Reddit, Discord, websites, and open-source datasets) during Dec 2022 to Dec 2023. Among these prompts, they identify 1,405 jailbreak prompts. To the best of our knowledge, this dataset serves as the largest collection of in-the-wild jailbreak prompts.

The data are provided here.

Statistics of our data source. (Adv) UA refers to (adversarial) user accounts.

License

jailbreak_llms is licensed under the terms of the MIT license. See LICENSE for more details.

Citation

If you find this useful in your research, please consider citing:

@inproceedings{SCBSZ24,
      author = {Xinyue Shen and Zeyuan Chen and Michael Backes and Yun Shen and Yang Zhang},
      title = {{``Do Anything Now'': Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models}},
      booktitle = {{ACM SIGSAC Conference on Computer and Communications Security (CCS)}},
      publisher = {ACM},
      year = {2024}
}