dataset_info:
- config_name: mrq
features:- name: question
dtype: string - name: choices
sequence: string - name: answers
sequence: int64 - name: categories
sequence: string
splits: - name: train
num_bytes: 2196394
num_examples: 3840
download_size: 510955
dataset_size: 2196394
- name: question
- config_name: prompts
features:- name: prompt
dtype: string - name: categories
sequence: string - name: source
dtype: string
splits: - name: base
num_bytes: 3992062
num_examples: 21318 - name: attackEnhanced
num_bytes: 13632986
num_examples: 5000 - name: defenseEnhanced
num_bytes: 61845
num_examples: 200
download_size: 4264443
dataset_size: 17686893
configs:
- name: prompt
- config_name: mrq
data_files:- split: train
path: mrq/train-*
- split: train
- config_name: prompts
data_files:- split: base
path: prompts/base-* - split: attackEnhanced
path: prompts/attackEnhanced-* - split: defenseEnhanced
path: prompts/defenseEnhanced-*
license: apache-2.0
language:
- split: base
- en
tags: - Safety
- AIGC
- LLM Safety
- Jailbreak
- Question-Answer
- Multiple Choice
pretty_name: Salad
size_categories: - 10K<n<100K
task_categories: - text-classification
- text-generation
Dataset Card for SaladBench
Paper: SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models
Data: SafeText Dataset
📊 Statistical Overview of Base Question
| Type | Data Source | Nums |
|---|---|---|
| Self-instructed | Finetuned GPT-3.5 | 15,433 |
| Open-Sourced | HH-harmless | 4,184 |
| HH-red-team | 659 | |
| Advbench | 359 | |
| Multilingual | 230 | |
| Do-Not-Answer | 189 | |
| ToxicChat | 129 | |
| Do Anything Now | 93 | |
| GPTFuzzer | 42 | |
| Total | 21,318 |
You can refer to the Paper, Code, Project Page for more related information.
License
- Licensed under Apache License 2.0
📝Citation
If you use SaladBench in your research, please cite the paper:
@article{li2024salad,
title={SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models},
author={Li, Lijun and Dong, Bowen and Wang, Ruohui and Hu, Xuhao and Zuo, Wangmeng and Lin, Dahua and Qiao, Yu and Shao, Jing},
journal={arXiv preprint arXiv:2402.05044},
year={2024}
}