license: mit
task_categories:

• text-generation
  tags:
• jailbreak
• safety
• red-teaming
  dataset_info:
• config_name: default
  features:
  • name: prompt
    dtype: string
  • name: prompt_name
    dtype: string
  • name: task
    dtype: string
  • name: category
    dtype: string
  • name: sub_category
    dtype: string
  • name: severity
    dtype: string
    splits:
  • name: train
    num_bytes: 2875375
    num_examples: 2250
    download_size: 289516
    dataset_size: 2875375
• config_name: prompts_only
  features:
  • name: prompt_template
    dtype: string
  • name: prompt_name
    dtype: string
    splits:
  • name: train
    num_bytes: 10349
    num_examples: 10
    download_size: 13427
    dataset_size: 10349
• config_name: tasks_only
  features:
  • name: task
    dtype: string
  • name: category
    dtype: string
  • name: sub_category
    dtype: string
  • name: severity
    dtype: string
    splits:
  • name: train
    num_bytes: 40250
    num_examples: 225
    download_size: 14215
    dataset_size: 40250
    configs:
• config_name: default
  data_files:
  • split: train
    path: tasks_within_prompts.parquet
• config_name: prompts_only
  data_files:
  • split: train
    path: prompts_only.parquet
• config_name: tasks_only
  data_files:
  • split: train
    path: tasks_only.parquet

Harmful Tasks Dataset

Dataset of 2250 harmful tasks and jailbreaking prompts for AI safety research.

Citation

@inproceedings{hasan2024pruning,
  title        = {Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning},
  author       = {Adib Hasan and Ileana Rugina and Alex Wang},
  booktitle    = {Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, co-located with EMNLP 2024},
  year         = {2024},
  address      = {Miami, USA},
  note         = {arXiv:2401.10862},
  url          = {https://arxiv.org/abs/2401.10862}
}

Usage

Access Data

from datasets import load_dataset

# Load from HuggingFace Hub (default: tasks_within_prompts)
dataset = load_dataset("notadib/harmful-tasks")

# Load specific configuration
prompts_only = load_dataset("notadib/harmful-tasks", "prompts_only")
tasks_only = load_dataset("notadib/harmful-tasks", "tasks_only")

# Access data
print(dataset['train'][0])

Evaluate Your Model

The classifier.py script evaluates jailbreak and refusal rates using three models:

• Target model: The model being tested for jailbreak resistance
• Jailbreak judge: Determines if responses contain harmful content
• Refusal judge: Determines if the model refused the request

All models use OpenAI-compatible APIs with separate base URLs and API keys. We recommend using Meta-Llama-3.3-70B-Instruct as the jailbreak and refusal judges for consistency.

# pip install tqdm openai pandas

./classifier.py \
  --target-model llama-3-8b \
  --target-base-url http://localhost:8000/v1 \
  --target-api-key YOUR_KEY \
  --jailbreak-model meta-llama/Meta-Llama-3.3-70B-Instruct \
  --jailbreak-base-url http://localhost:8001/v1 \
  --jailbreak-api-key YOUR_KEY \
  --refusal-model meta-llama/Meta-Llama-3.3-70B-Instruct \
  --refusal-base-url http://localhost:8001/v1 \
  --refusal-api-key YOUR_KEY \
  --out results.json \
  --max-concurrent 16

The --max-concurrent argument (default: 16) limits parallel API requests.

Results include:

• total_prompts: Number of prompts evaluated (2250)
• attack_success_rate: Fraction of jailbreak attempts that succeeded
• refusal_rate: Fraction of requests that were refused
• samples: Per-prompt details (prompt, response, jailbroken, refused)

Dataset Splits

This dataset provides three different configurations:

1. tasks_within_prompts (default)

Combines jailbreaking prompts with harmful tasks by replacing <HARMFUL TASK> placeholder in prompt templates.

Features:

• prompt: The full prompt with task inserted
• prompt_name: Name of the jailbreaking prompt technique (e.g., "AIM", "DAN")
• task: The harmful task
• category: Task category (e.g., "Misinformation and Disinformation")
• sub_category: Task sub-category (e.g., "Social Media Rumors and Misleading Posts")
• severity: Task severity level ("Low Severity", "Medium Severity", "High Severity")

2. prompts_only

Contains only jailbreaking prompt templates without tasks.

Features:

• prompt_template: The prompt template with <HARMFUL TASK> placeholder
• prompt_name: Name of the jailbreaking prompt technique

3. tasks_only

Contains only harmful tasks without jailbreaking prompts.

Features:

• task: The harmful task
• category: Task category
• sub_category: Task sub-category
• severity: Task severity level

Dataset Statistics

• 10 jailbreaking prompt templates
• 225 unique harmful tasks
• 2,250 prompt-task combinations in tasks_within_prompts

Ethical Considerations

This dataset is intended solely for AI safety research, including:

• Developing robust content moderation systems
• Testing AI model safety and alignment
• Creating defensive security tools
• Understanding adversarial prompt techniques

Do not use this dataset for malicious purposes.