license: mit
task_categories:
- text-generation
language: - en
tags: - safety
- refusal-training
- openelm
OpenELM Safety Training Dataset
A curated dataset of 3,094 examples for fine-tuning OpenELM-1.1B with robust safety behavior.
📊 Dataset Composition
| Category | Count | Percentage |
|---|---|---|
| Harmful (Refusal) | 1,094 | 35.4% |
| Benign (Compliance) | 2,000 | 64.6% |
| Total | 3,094 | 100% |
Harmful:Benign Ratio: 1:1.8
📁 Data Sources
Harmful Prompts (Base: 912 unique, augmented to 1,094)
| Source | Dataset ID | Split | Count | Notes |
|---|---|---|---|---|
| AdvBench | walledeval/AdvBench |
train | 520 | Via JailbreakBench loader |
| HarmBench Standard | walledai/HarmBench |
standard / train |
200 | Standard harmful behaviors |
| HarmBench Contextual | walledai/HarmBench |
contextual / train |
100 | Context-dependent harms |
| JailbreakBench | JailbreakBench/jbb-behaviors |
harmful |
100 | Jailbreak goal prompts |
| HarmBench Extra | mariagrandury/harmbench |
DirectRequest / test |
80 | Additional behaviors |
After deduplication: 912 unique harmful prompts
Benign Prompts (2,000 total)
| Source | Dataset ID | Split | Count |
|---|---|---|---|
| Alpaca | tatsu-lab/alpaca |
train (first 2,000) | 2,000 |
🔄 Augmentations Applied
| Augmentation | Ratio | Description |
|---|---|---|
| Standard Refusal | 60% | Direct refusal + safe alternative |
| DeRTa (Defensive Reframing) | 20% | Acknowledges topic, pivots to safe educational context |
| Past-Tense Variants | 20% | Historical framing of harmful prompts (e.g., "How did people...") |
Total augmented harmful examples: 912 base + 182 past-tense = 1,094
🎯 Refusal Categories
Prompts are categorized using keyword heuristics for topic-aligned refusals:
| Category | Count | Example Alternative |
|---|---|---|
GENERAL |
595 | Generic safety refusal |
CYBER/HACKING |
172 | Defensive security, blue teaming |
VIOLENCE/Harm |
98 | Crisis resources, safety information |
DRUGS |
25 | Health effects, addiction support |
COPYRIGHT |
22 | Summary, analysis, cultural context |
🤖 Refusal Generation
All 1,094 refusals were dynamically generated using:
- Model:
llama-3.1-8b-instantvia Groq API - Method: Proactive round-robin key rotation across 5 API keys
- Fallback Templates Used: 0 (verified programmatically)
System prompts enforced strict topic alignment to prevent "domain drift" (e.g., drug prompts do NOT pivot to cybersecurity lectures).
⚠️ Disclaimers
Copyright Category: Includes refusals for copyrighted content (lyrics, books). These are distinct from physical harm refusals.
Benign Data Accuracy: Benign samples from Alpaca are not independently fact-checked. Some may be outdated or oversimplified.
Not a Safety Benchmark: This dataset is for training, not evaluation. For evaluation, use HarmBench or JailbreakBench directly.
📂 Files
| File | Description |
|---|---|
training_data.json |
Full dataset (3,094 examples) |
training_sample.json |
Sample of ~8 examples per category |
📜 License
MIT License. See repository root for details.