dataset_info:
features:

• name: prompt
  dtype: string
• name: adversarial
  dtype: bool
• name: label
  dtype: string
  splits:
• name: train
  num_bytes: 856863
  num_examples: 1725
  download_size: 490550
  dataset_size: 856863
  configs:
• config_name: default
  data_files:
  • split: train
    path: data/train-*
    license: odc-by
    tags:
• safe
• safety
• jailbreak
• ai-safety
• llm
• lm
• moderation
• classification
• refusal
  task_categories:
• text-classification
  language:
• en
  size_categories:
• 10K<n<100K
  extra_gated_prompt: >-
  Access to this dataset is automatically granted upon accepting the AI2 Responsible Use Guidelines, and completing
  all fields below
  extra_gated_fields:
  Your full name: text
  Organization or entity you are affiliated with: text
  State or country you are located in: text
  Contact email: text
  Please describe your intended use of the low risk artifact(s): text
  I understand that this dataset is a research artifact that may contain or produce unfiltered, toxic, or harmful material: checkbox
  I agree to use this dataset for research purposes in accordance with the AI2 Responsible Use Guidelines: checkbox
  I agree that AI2 may use my information as described in the Privacy Policy: checkbox
  I certify that the information I have provided is true and accurate: checkbox

Dataset Card for WildGuardMix

Paper: WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs

Data: WildGuardMix Dataset

Disclaimer

The data includes examples that might be disturbing, harmful, or upsetting. It covers discriminatory language, discussions about abuse, violence, self-harm, sexual content, misinformation, and other high-risk categories. It is recommended not to train a Language Model exclusively on the harmful examples.

Dataset Summary

• Data Size: 1,725 items for prompt harm, response harm, and response refusal classification tasks
• Data Types: Vanilla and adversarial synthetic data, in-the-wild user-LLLM interactions
• Labels: Annotated by three independent annotators, with Fleiss Kappa scores indicating moderate to substantial agreement
• Labels Quality: Further validated using prompted GPT-4 classifier and manual inspection

Please refer to the paper for detailed data construction methods.

Citation

If you use WildGuardTest in your research, please cite the paper:

@misc{wildguard2024,
      title={WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs}, 
      author={Seungju Han and Kavel Rao and Allyson Ettinger and Liwei Jiang and Bill Yuchen Lin and Nathan Lambert and Yejin Choi and Nouha Dziri},
      year={2024},
      eprint={2406.18495},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2406.18495}, 
}