language:
- en
- vi
- zh
task_categories: - text2text-generation
pretty_name: Categorical Harmful QA
dataset_info:
features:- name: prompt
dtype: string - name: category
dtype: string - name: subcategory
dtype: string
splits: - name: en
num_bytes: 86873
num_examples: 550 - name: zh
num_bytes: 75685
num_examples: 550 - name: vi
num_bytes: 120726
num_examples: 550
download_size: 95186
dataset_size: 283284
configs:
- name: prompt
- config_name: default
data_files:- split: en
path: data/en-* - split: zh
path: data/zh-* - split: vi
path: data/vi-*
license: apache-2.0
- split: en
Dataset Card for CatQA
Data: CatQA Dataset
About
CatQA is used in LLM safety realignment research as a categorical harmful questions dataset. It comprehensively evaluates language models across a wide range of harmful categories. The dataset includes questions from 11 main categories of harm, each divided into 5 sub-categories, totaling 550 harmful questions. CatQA is available in English, Chinese, and Vietnamese to assess generalizability.
For more details, please refer to the paper and the GitHub repository.
License
- Licensed under Apache License 2.0
Citation
If you use CatQA in your research, please cite the paper:
@misc{bhardwaj2024language,
title={Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic},
author={Rishabh Bhardwaj and Do Duc Anh and Soujanya Poria},
year={2024},
eprint={2402.11746},
archivePrefix={arXiv},
primaryClass={cs.CL}
}