license: cc-by-nc-sa-4.0
task_categories:

• text-classification
  language:
• en
  tags:
• safety
• alignment
• content-moderation
• guardrail
• supervision
  pretty_name: BELLS-O Content Moderation Input Dataset
  size_categories:
• 1K<n<10K
  language_creators:
• found
• expert-generated
• machine-generated
  source_datasets:
• extended|brash6/content_moderation_bells_v2
• extended|Bertievidgen/SimpleSafetyTests
• extended|sorry-bench/sorry-bench-202503
• extended|Paul/XSTest
• extended|LLM-LAT/benign-dataset
  multilinguality:
• multilingual
  extra_gated_prompt: You agree not to use the dataset or information included in it to cause harm.
  extra_gated_fields:
  "Name of the Company or Institution you are affiliated with, or Independent": text
  "Institutional/Organizational email address":
  type: text
  optional: true
  Country: country
  Describe your intended use in 2-3 sentences (insufficient detail = declined): text
  I agree to use this dataset for non-commercial use ONLY: checkbox
  "I read the 'Access Guidelines' section in the README": checkbox

Access Guidelines - READ THIS BEFORE REQUESTING ACCESS!

Access is only granted to identifiable individuals with proper reason to use this sensitive data.

If any other dataset could be used to accomplish your goal, this does not count as a proper reason. Half sentences and bullet points do not suffice and will be declined. Proper reasons include anything that showcases your specific need for this exact dataset.

Content Moderation Dataset

Overview

This dataset contains 1,400 prompts designed to evaluate supervision systems on content moderation tasks as part of the BELLS-Operational benchmark. It enables rigorous assessment of LLM safety mechanisms across detection accuracy, false positive rates, latency, and cost.

Dataset Statistics

Multilinguality Disclaimer

The majority of prompts (99%) are in English. Only some of the OpenAI samples in the Sexual Content category are in different languages.

Taxonomy

Our taxonomy covers 11 harmful categories aligned with EU AI Act systemic risk categories and GPAI responsible use policies and 1 benign category to measure the harmful refusal rate:

Harmful Categories (11)

1. CBRN: Chemical, biological, radiological, nuclear weapons and dangerous materials
2. Cyber: Malware, hacking, exploits, and cybersecurity threats
3. Harm to Minors: Content endangering, exploiting, or sexualizing individuals under 18
4. Harmful Manipulation: Scams, fraud, phishing, psychological manipulation, misinformation
5. Hate Speech: Discrimination, violence, or dehumanization based on protected characteristics
6. Illegal Activities: Drug trafficking, weapons trafficking, financial crimes, human trafficking
7. Integrity & Quality: Academic dishonesty, plagiarism, credential falsification, spam
8. Physical Harm: Violence instructions, weapons, explosives, dangerous activities
9. Privacy: Doxxing, unauthorized surveillance, personal information disclosure, stalking
10. Self-Harm: Suicide, eating disorders, self-injury promotion or instructions
11. Sexual Content: Explicit material, pornography, non-consensual intimate content

Benign Category (1)

Benign: Legitimate requests covering cooking, education, travel, technology, creative writing, and general knowledge. Critical for measuring false positive rates.

Methodology

Construction Strategy

We used three complementary data sources to ensure diversity, realism, and comprehensive coverage:

1. Extracted from Existing Datasets (620 prompts, 44%)

Purpose: Leverage validated real-world examples from established safety benchmarks.

Advantages:

• Empirically grounded test cases from production systems
• External validity and comparability with prior work
• Authentic adversarial attack patterns

Distribution:

• Standard: 40 prompts per harmful category (when available)
• Benign: 200 prompts for robust FPR baseline

2. Handcrafted Prompts (400 prompts, 29%)

Purpose: Capture edge cases, nuanced scenarios, and adversarial intensities that automated methods miss.

Process:

• Manually designed by AI safety researchers
• Focus on boundary cases and sophisticated evasion attempts
• Iteratively refined through expert review

Distribution:

• Standard: 30 prompts per harmful category
• Benign: 100 prompts for diverse legitimate use cases
• Harm to Minors: 0 prompts (ethical decision to avoid creating potentially harmful content)

3. AI-Generated Prompts (380 prompts, 27%)

Purpose: Scale coverage and fill gaps in categories with limited public data or sensitive content.

Process:

• Adversarial generation with multi-stage filtering pipelines
• Quality control through automatic filtering and manual review
• Variation across adversarial intensity levels

Distribution:

• Standard: 30 prompts per harmful category
• No generation for Benign category
• Harm to Minors: 70 prompts (compensating for 0 handcrafted due to ethical concerns)
• Integrity & Quality: 40 prompts (limited high-quality extracted data available)

Design Rationale

Standard Split: 30-40-30

Our baseline distribution follows 30 AI-generated, 40 extracted, 30 handcrafted:

• Maximizes validated real-world data (40 extracted)
• Supplements with targeted edge cases (30 handcrafted)
• Adds scalable diversity (30 AI-generated)

Deviations from Standard

Harm to Minors (70-30-0):

• Ethical decision: No handcrafted prompts to avoid researcher exposure to child safety violations
• Compensated with 70 AI-generated (filtered) + 30 extracted from vetted datasets

Integrity & Quality (40-30-30):

• Increased AI generation due to limited public datasets for this emerging category
• Covers academic dishonesty, misinformation generation, credential falsification

Category Distribution

100 prompts per harmful category:

• Balanced evaluation across categories
• Sufficient sample size for statistical significance
• Prevents single category from dominating metrics

300 benign prompts (3× harmful categories):

• False positive rate is critical operational metric requiring precision
• Benign content spans vastly more diverse use cases
• Over-refusal directly impacts system utility and user trust

Ground Truth Philosophy

Labels are deliberately conservative:

• Harmful: Clear policy violations with minimal ambiguity
• Benign: Unambiguously acceptable requests
• Edge cases excluded: Ensures reproducible evaluation without subjective judgment

Purpose

Evaluate supervision systems on content moderation under operational constraints.

Key Evaluation Dimensions

1. Detection Rate: Percentage of harmful prompts correctly identified
2. False Positive Rate: Percentage of benign prompts incorrectly flagged
3. Latency: End-to-end response time in realistic deployment scenarios
4. Cost: Per-prompt API costs or estimated compute expenses

Source Datasets

Our extracted prompts (620, 44%) draw from:

• Benign Dataset - Used for the extracted benign samples.
• Content Moderation Dataset Bells V2 - Dataset used in the previous Bells iteration
• HarmBench - Standardized red teaming evaluation (Mazeika et al., 2024)
• OpenAI Moderation Dataset - Content moderation evaluation examples
• SimpleSafetyTests - Simple safety test cases for LLM evaluation
• SORRY-Bench - Systematic LLM safety refusal evaluation (Xie et al., 2025)
• XSTest - Over-refusal and exaggerated safety tests

Some samples from the Bells V2 dataset are originally from other datasets:

• AdvBench - Adversarial attacks on aligned language models
• Anthropic HH-RLHF - Red teaming attempts from human feedback
• CatQA - Categorized question-answering safety dataset
• DAN (Do Anything Now) - Jailbreak prompts and adversarial attacks collection
• Do Not Answer - Dataset for evaluating safeguards against harmful instructions
• JailbreakBench - Benchmark for jailbreak attacks on LLMs

Paraphrasing to Remove Model Fingerprints

The Problem: During initial evaluation, Mistral models consistently outperformed all other models on the benchmark, achieving 96% accuracy. This performance seemed suspiciously high and suggested the dataset retained distinctive stylistic fingerprints from the Mistral generation model used to create the responses. This creates unwanted bias where the generation model has an unfair advantage over other models being evaluated, as supervision systems may have an advantage by having a similar internal representation space as the generating model.

The Solution: To remove model-specific fingerprints and ensure fair evaluation, we paraphrased all 1,400 generated responses using an older model architecture (Llama 2 / Vicuna 13B) that is no longer actively deployed in production systems. By using a model that evaluators are unlikely to encounter in practice, we ensure that evaluated supervision systems aren't biased toward recognizing the generation model's distinctive style. The paraphrasing process maintains the semantic content and meaning of each response while removing the characteristic patterns that could identify the source model.

Implementation Challenges: Initial attempts to use base models from the Llama 2 to Vicuna family (including larger 30B+ parameter variants) revealed poor instruction-following capabilities for paraphrasing tasks. These base models struggled to preserve semantic meaning while modifying stylistic elements, often producing outputs that were either too similar to the input (failing to remove fingerprints) or too different (losing important semantic content). To address this challenge, we fine-tuned Vicuna 13B specifically for the paraphrasing task.

Fine-tuning Approach:

• Training data: We curated 7,000 high-quality samples combining existing human conversation datasets with our own manually written samples to ensure diversity and naturalness
• Training method: We used the Mistral 24B model to paraphrase human-written text, creating (human text, Mistral paraphrase) pairs. We then trained Vicuna 13B on the inverse function to learn the mapping from Mistral output back to human-like text
• Training technique: LoRA (Low-Rank Adaptation) fine-tuning for parameter-efficient training
• Validation: We tested the fine-tuned model against multiple AI detection systems to ensure outputs were indistinguishable from human writing

Result: The fine-tuned paraphrasing model successfully produces human-like output with natural imperfections including grammatical errors, typos, and stylistic variations typical of human writing. All paraphrased samples passed AI detection tests and were classified as human-written text. This paraphrasing approach successfully removed Mistral fingerprints while maintaining the semantic content, sample quality, and naturalness of the responses. Post-paraphrasing evaluation showed that Mistral models no longer exhibited the suspicious 96% accuracy advantage, confirming successful removal of model-specific fingerprints.

Contact

• Leonhard Waibl: leonhard.waibl{at}student[dot]tugraz.at
• Felix Michalak: felix[at]michalax{.}de
• Hadrien Mariaccia: hadrien{at}securite-ia[dot]fr

Citation

If you use this dataset in your research, please cite:

@dataset{bells_content_moderation_input_dataset_2026,
  title={BELLS-O: Content Moderation Input Dataset},
  author={Waibl, Leonhard and Michalak, Felix and Mariaccia, Hadrien},
  year={2026},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/datasets/centrepourlasecuriteia/content-moderation-input-dataset}}
}

Part of BELLS-Operational • SPAR Fall 2025 • CeSIA