license: cc-by-4.0
task_categories:

• text-generation
  language:
• en
  tags:
• safety
• red-teaming
• jailbreak
• llm-agent
• mcp
• gated
  pretty_name: FragBench (Restricted)
  version: "1.0.0"
  size_categories:
• n<1K
  citation: |
  @inproceedings{fragbench2027,
  title = {FragBench: A Benchmark for Detecting Fragmented Multi-Session Jailbreak Attacks on LLM Agents},
  author = {Anonymous},
  booktitle = {International Conference on Learning Representations (under review)},
  year = {2027}
  }
  extra_gated_prompt: >-
  This dataset contains red-teaming system prompts and reinforcement-learning
  attack scaffolds that can be used to elicit harmful behaviour from large
  language models. Access is restricted to vetted safety researchers.

By requesting access you confirm that:
1. You will use the data solely for academic safety research, evaluation,
or red-team defence development.
2. You will not redistribute the data or derived models.
3. You will not use the data to attack production systems or third
parties without authorisation.
4. You will comply with the CC-BY-4.0 license.
extra_gated_fields:
Full name: text
Affiliation: text
Institutional email: text
Intended use (2-3 sentences): text
I agree not to redistribute: checkbox
I agree to use only for safety research: checkbox
extra_gated_button_content: Request access
configs:

• config_name: rl_runs
  data_files:
  • split: train
    path: "data/rl_runs.parquet"
    default: true

FragBench (Restricted Tier)

Anonymous submission for ICLR 2027.
Author identity will be revealed at camera-ready.

Companion public tier

This restricted tier contains only the sensitive components: RL system
prompts, judge-rubric configurations, and high-yield variant traces.

The seed campaigns, generated variants, and benign data are in the public
companion dataset, which is freely accessible without request-access:

https://huggingface.co/datasets/anon-fragbench-neurips/fragbench

Most users will want both. Approved researchers should download the public
dataset (no gating) and this restricted dataset together.

Dataset Summary

This is the gated companion to the public FragBench dataset. It contains
the more sensitive components of the FragBench Attack pipeline:

• Reinforcement-learning system prompts used by the attack rewriter
• Judge-rubric configurations and elicitation scaffolds
• High-yield variant traces selected by the RL loop

The non-sensitive seed campaigns, generated variants, and benign data are
in the public companion dataset.

Dataset Structure

fragbench-restricted/
└── rl-outputs-manual/        # RL run outputs, one folder per (campaign × judge × rewriter × verifier) configuration
                              # folder naming convention:
                              #   <campaign>_manual__rl__<verifier>__judge-<judge_model>__rewriter-<rewriter_model>__max-<N>__<timestamp>
                              # each folder contains the full RL trajectory for that run
                              # (rewritten variants, judge verdicts, selected high-yield outputs)

Approximately 360 files across the model and verifier sweep (~58 MB total).
Folder names encode every hyperparameter so a single run can be reproduced
or filtered by judge / rewriter / campaign without opening individual files.

Examples of folder names you will see:

• dprk_fraud_manual__rl__vs-command_form__judge-claude-opus-4-6__rewriter-claude-sonnet-4-6__max-20__20260506-215210
• <campaign>_manual__rl__<verifier>__judge-claude-opus-4-6__rewriter-deepseek_deepseek-v4-flash__max-20__<timestamp>

Why this tier is gated

These artefacts are direct attack tooling. Releasing them openly would
materially lower the barrier to attacking deployed LLM agents. We follow
the standard practice for security datasets (HarmBench, JailbreakBench)
of placing such components behind request-access.

Access policy

Requests are reviewed manually. We approve users who:

• Have an institutional affiliation (academic, government, or industry
  safety research)
• Provide a clear, specific intended use related to LLM safety research
• Agree to non-redistribution and non-commercial use

Approval typically takes 1–5 business days.

Considerations for Using the Data

Intended uses

• Reproducing the FragBench Attack pipeline for evaluation
• Training and red-teaming defensive classifiers
• Studying RL-based attack generation for safety research

Prohibited uses

• Direct deployment against production LLM agents or third-party services
• Redistribution in any form, including derived datasets or fine-tuned
  models trained primarily on this data
• Use that violates the attribution requirements of CC-BY-4.0

Risks

The prompts in this dataset have been observed to substantially increase
harmful-output rates on frontier LLMs. Treat them with care.

Licensing Information

CC-BY-4.0 + manual access gating (the gating is access control, not a license restriction).
See LICENSE in this repository.

Citation

@inproceedings{fragbench2027,
  title  = {FragBench: A Benchmark for Detecting Fragmented Multi-Session
            Jailbreak Attacks on LLM Agents},
  author = {Anonymous},
  booktitle = {International Conference on Learning Representations (under review)},
  year   = {2027}
}