dataset_info:

• config_name: DirectRequest
  features:
  • name: Behavior
    dtype: string
  • name: FunctionalCategory
    dtype: string
  • name: SemanticCategory
    dtype: string
  • name: Tags
    dtype: string
  • name: ContextString
    dtype: string
  • name: BehaviorID
    dtype: string
    splits:
  • name: val
    num_bytes: 39120
    num_examples: 80
  • name: test
    num_bytes: 165190
    num_examples: 320
    download_size: 127291
    dataset_size: 204310
• config_name: HumanJailbreaks
  features:
  • name: BehaviorID
    dtype: string
  • name: Behavior
    dtype: string
    splits:
  • name: val
    num_bytes: 919055
    num_examples: 400
  • name: test
    num_bytes: 3854255
    num_examples: 1600
    download_size: 2126455
    dataset_size: 4773310
    configs:
• config_name: DirectRequest
  data_files:
  • split: val
    path: DirectRequest/val-*
  • split: test
    path: DirectRequest/test-*
• config_name: HumanJailbreaks
  data_files:
  • split: val
    path: HumanJailbreaks/val-*
  • split: test
    path: HumanJailbreaks/test-*

HarmBench

• Original repo: https://github.com/centerforaisafety/HarmBench/tree/main/data/behavior_datasets

This HF dataset contains two types of harmful prompts:

• DirectRequest: taken from harmbench_behaviors_text_test.csv (test split) and harmbench_behaviors_text_val.csv (val split)
• HumanJailbreaks: manual selection of jailbreaks divided into test and val splits
• Moreover, metadata.csv contains the information about each behavior and can be mapped to the prompts above by BehaviourID (original_file)

How to load the data:

load_dataset("swiss-ai/harmbench", "DirectRequest", split="test")
load_dataset("swiss-ai/harmbench", "HumanJailbreaks", split="test")