license: cc-by-nc-4.0
🧩 Moderated Jailbreak Template Dataset
This dataset provides 100 pairs of original jailbreak prompts and their moderated counterparts. Each moderated template retains the structure of its jailbreak origin but has been safely rewritten to eliminate any harmful or unethical content.
🧠 What is a Moderated Template?
A Moderated Template is a prompt that looks like a jailbreak in form, tone, and structure — but is semantically safe and does not elicit harmful responses.
These prompts are designed to evaluate whether a jailbreak detector can distinguish between:
- truly malicious prompts, and
- benign prompts that resemble adversarial inputs.
This helps identify false positive vulnerabilities in prompt-level defense systems.
📂 Dataset Structure
| Column | Description |
|---|---|
category |
Type of jailbreak strategy (e.g., roleplay, token manipulation, etc.) |
name |
Name or identifier of the jailbreak template |
jailbreak |
Original jailbreak prompt |
moderation |
Moderated version aligned with safety guidelines, preserving structure |
🎯 Why This Dataset Matters
Moderated Templates are critical for:
- Evaluating and minimizing false positives in jailbreak detectors
- Developing context-aware and nuanced defense models
- Balancing usability and safety in LLM applications
Most existing detectors fail to differentiate structurally deceptive but benign prompts — this dataset fills that gap.
🛠️ Construction Details
Original Prompts: 100 jailbreak templates were used:
- 78 templates from the benchmark in:
Liu, Y. et al. Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study
- 22 templates collected from public communities (e.g., Reddit).
- 78 templates from the benchmark in:
Moderation Process: Moderated templates were generated using GPT-4o, with human review and refinement. Unsafe content was replaced with contextually appropriate alternatives while maintaining the original structure.
Goal: Enable researchers to test whether pre-detectors can avoid false positives when encountering structurally adversarial-looking but ethically sound prompts.
🔐 Access
This dataset is private due to potential misuse.
To request access, click “Request Access” on the dataset page. We manually review requests for research purposes only.