license: other
license_name: hex-phi
license_link: https://huggingface.co/datasets/LLM-Tuning-Safety/HEx-PHI/#hex-phi-dataset-license-agreement
extra_gated_prompt: You agree to the HEx-PHI Dataset License Agreement. Also, please specify the following fields in detail (we suggest you fill in your affiliation email), based on which we will inspect and manually grant access to approved users. If you have not been granted access, please email us (see email contact from our paper) and specify more details.
extra_gated_fields:
Name: text
Email: text
Affiliation: text
Country: text
Purpose: text
configs:

• config_name: default
  data_files:
  • split: Category_1_Illegal_Activity
    path: category_1.csv
  • split: Category_3_Hate_Harass_Violence
    path: category_3.csv
  • split: Category_4_Malware
    path: category_4.csv
  • split: Category_5_Physical_Harm
    path: category_5.csv
  • split: Category_6_Economic_Harm
    path: category_6.csv
  • split: Category_7_Fraud_Deception
    path: category_7.csv
  • split: Category_8_Adult_Content
    path: category_8.csv
  • split: Category_9_Political_Campaigning
    path: category_9.csv
  • split: Category_10_Privacy_Violation_Activity
    path: category_10.csv
  • split: Category_11_Tailored_Financial_Advice
    path: category_11.csv
    task_categories:
• text-generation
• conversational
  language:
• en
  pretty_name: Human-Extended Policy-Oriented Harmful Instruction Benchmark
  size_categories:
• n<1K
  tags:
• harmfulness
• benchmark

HEx-PHI: Human-Extended Policy-Oriented Harmful Instruction Benchmark

This dataset contains 330 harmful instructions (30 examples x 11 prohibited categories) for LLM harmfulness evaluation.

In our work "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!", to comprehensively cover as many harmfulness categories as possible,
we develop this new safety evaluation benchmark directly based on the exhaustive lists of prohibited use cases found in Meta’s Llama-2 usage policy and OpenAI’s usage policy.
Specifically, we gather 30 examples of harmful instructions for each of the 11 prohibited use case categories from various sources, including:

• Anthropic Red Teaming Data
• AdvBench from "Universal and Transferable Adversarial Attacks on Aligned Language Models"
• The harmful requests are further categorized & refined by human annotators, GPT-4, Claude, jailbroken GPT-3.5, jailbroken Llama2, etc.

For more details, please refer to our project website: http://llm-tuning-safety.github.io.

Revision (Aug 19th, 2024)

🚨 In this revision, we have removed Chile Sexual Abuse Material (CSAM) in our data to avoid negative societal impact. In particular, we:

• Editted the 29th record in category_1.csv, which was related to CSAM.
• Removed all 30 harmful instructions from the the 2nd category (Child Abuse Content).

To access the original dataset, please contact the authors.

Dataset Format

The 11 categories of harmful instructions are stored separately in 11 independent files in plain CSV format:

• category_1.csv: Illegal Activity
• category_2.csv: Child Abuse Content (🚨As per Aug 19th 2024 revision, we have removed this file from our repository to avoid spreading CASM. Please contact the authors to access this category.)
• category_3.csv: Hate / Harass /Violence
• category_4.csv: Malware
• category_5.csv: Physical Harm
• category_6.csv: Economic Harm
• category_8.csv: Fraud Deception
• category_7.csv: Adult Content
• category_9.csv: Political Campaigning
• category_10.csv: Privacy Violation Activity
• category_11.csv: Tailored Financial Advice

Dataset Usage

With our 11 harmful categories, we hope HEx-PHI can help researchers comprehensively evaluate fine-grained safety risks and harmfulness underlying LLMs.
Practically, one should use HEx-PHI harmful instructions as (part of) input prompts, and inspect👀 whether the responses generated by the LLM satisfy the harmful intentions of the instructions.

👀In our paper, we use GPT-4 as the harmfulness judge to assign each <user instruction, model response> pair a harmfulness score from 1 to 5. Refer to Appendix B for details.

HEx-PHI Dataset License Agreement

This Agreement contains the terms and conditions that govern your access and use of the HEx-PHI Dataset (as defined above). You may not use the HEx-PHI Dataset if you do not accept this Agreement. By clicking to accept, accessing the HEx-PHI Dataset, or both, you hereby agree to the terms of the Agreement. If you are agreeing to be bound by the Agreement on behalf of your employer or another entity, you represent and warrant that you have full legal authority to bind your employer or such entity to this Agreement. If you do not have the requisite authority, you may not accept the Agreement or access the HEx-PHI Dataset on behalf of your employer or another entity.

• Safety and Moderation: This dataset contains unsafe conversations or prompts that may be perceived as offensive or unsettling. Users may not use this dataset for training machine learning models for any harmful purpose. The dataset may not be used to generate content in violation of any law. These prompts should not be used as inputs to models that can generate modalities outside of text (including, but not limited to, images, audio, video, or 3D models)
• Non-Endorsement: The views and opinions depicted in this dataset do not reflect the perspectives of the researchers or affiliated institutions engaged in the data collection process.
• Legal Compliance: You are mandated to use it in adherence with all pertinent laws and regulations.
• Model Specific Terms: When leveraging direct outputs of a specific model, users must adhere to its corresponding terms of use and relevant legal standards.
• Non-Identification: You must not attempt to identify the identities of individuals or infer any sensitive personal data encompassed in this dataset.
• Prohibited Transfers: You should not distribute, copy, disclose, assign, sublicense, embed, host, or otherwise transfer the dataset to any third party.
• Right to Request Deletion: At any time, we may require you to delete all copies of this instruction dataset (in whole or in part) in your possession and control. You will promptly comply with any and all such requests. Upon our request, you shall provide us with written confirmation of your compliance with such requirement.
• Termination: We may, at any time, for any reason or for no reason, terminate this Agreement, effective immediately upon notice to you. Upon termination, the license granted to you hereunder will immediately terminate, and you will immediately stop using the HEx-PHI Dataset and destroy all copies of the HEx-PHI Dataset and related materials in your possession or control.
• Limitation of Liability: IN NO EVENT WILL WE BE LIABLE FOR ANY CONSEQUENTIAL, INCIDENTAL, EXEMPLARY, PUNITIVE, SPECIAL, OR INDIRECT DAMAGES (INCLUDING DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION, OR LOSS OF INFORMATION) ARISING OUT OF OR RELATING TO THIS AGREEMENT OR ITS SUBJECT MATTER, EVEN IF WE HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Subject to your compliance with the terms and conditions of this Agreement, we grant to you, a limited, non-exclusive, non-transferable, non-sublicensable license to use the HEx-PHI Dataset, including the conversation data and annotations, to research, and evaluate software, algorithms, machine learning models, techniques, and technologies for both research and commercial purposes.

Citation

@inproceedings{
anonymous2024finetuning,
title={Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!},
author={Xiangyu Qi and Yi Zeng and Tinghao Xie and Pin-Yu Chen and Ruoxi Jia and Prateek Mittal and Peter Henderson},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=hTEGyKf0dZ}
}