license: odc-by
language:
- en
task_categories: - text-classification
tags: - safety
- harm-detection
- probing
- wildguardmix-derived
size_categories: - 1K<n<10K
extra_gated_prompt: >-
Access to this dataset is automatically granted upon accepting the
AI2 Responsible Use Guidelines,
and completing all fields below. This dataset is derived from
allenai/wildguardmix, which is released behind the same agreement; by
requesting access you agree to those guidelines for this derivative as well.
extra_gated_fields:
First name: text
Last name: text
Affiliation: text
Country: country
I agree to the AI2 Responsible Use Guidelines: checkbox
I agree to use this dataset for research purposes only: checkbox
configs: - config_name: default
data_files:- split: train
path: data/probe_training.parquet
- split: train
English harm-probe training set (WildGuardMix-derived)
⚠️ Content warning. Half of these prompts are harmful requests (violence,
fraud, self-harm, hate, criminal planning, and other harm categories). They are
released for safety research only. Reading them may be disturbing or upsetting.
Contents
A 50/50 harmful/benign English prompt set for training linear harm-detection
probes, filtered and subsampled from allenai/wildguardmix
(config wildguardtrain, split train, commit d29c47f41c8b51348b5c8e8c81c039b3132b66d1).
One file, data/probe_training.parquet — 5,593 unique prompts carrying two
overlapping 3,000-prompt training variants (407 prompts appear in both):
| column | type | meaning |
|---|---|---|
id |
string | stable item id (wgm_ + 12-hex content hash) |
prompt |
string | the English prompt, verbatim from WildGuardMix |
label |
int | 1 = harmful, 0 = benign (WildGuardMix prompt_harm_label) |
source |
string | always wildguardmix |
in_train |
bool | in the primary uniform-random training set (3,000 rows: 1,500/1,500) |
in_train_lenmatch |
bool | in the length-matched variant (3,000 rows: 1,500/1,500) |
The length-matched variant exists because harmful prompts are longer on average
(length-only AUC 0.760 in the uniform sample); it removes prompt length as a
confound when training probes.
Provenance
| upstream dataset | count | license |
|---|---|---|
allenai/wildguardmix (wildguardtrain) |
5,593 (100%) | ODC-BY 1.0, gated behind the AI2 Responsible Use Guidelines |
Filter chain (from 86,759 loaded rows): English only (langdetect) → vanilla
prompts only (adversarial == False) → prompt_harm_label mapped to a binary
label → ≤512 Gemma-3 tokens → decontaminated against our 200-item multilingual
evaluation set (0 exact + 1 fuzzy collision removed at token_sort_ratio ≥ 90) →
internal exact dedup (20,709 removed) → balanced 3,000-prompt samples
(seed 42). Full per-step counts ship in the companion code repository.
No translations are included in this repo; prompts are verbatim WildGuardMix
text (this dataset is redistribution-of-selection, not adaptation).
Intended use
Training and analyzing lightweight harm-detection probes (e.g., logistic
regression on LLM residual-stream activations) and other prompt-level safety
classifiers, for research. Per upstream guidance, do not train a generative
model exclusively on the harmful half.
Licensing and attribution
Released under ODC-BY 1.0 (https://opendatacommons.org/licenses/by/1-0/),
matching the upstream license. Contains information from
WildGuardMix (Allen Institute for AI), which is made available under the
ODC Attribution License. This repo is gated with the same
AI2 Responsible Use Guidelines
agreement as the upstream dataset.
Please cite WildGuard alongside our paper:
@misc{wildguard2024,
title = {WildGuard: Open One-Stop Moderation Tools for Safety Risks,
Jailbreaks, and Refusals of LLMs},
author = {Han, Seungju and Rao, Kavel and Ettinger, Allyson and Jiang, Liwei
and Lin, Bill Yuchen and Lambert, Nathan and Choi, Yejin and
Dziri, Nouha},
year = {2024},
eprint = {2406.18495},
archivePrefix = {arXiv}
}