license: apache-2.0
task_categories:
- text-generation
language: - en
tags: - red-teaming
- ai-safety
- jailbreak
- dpo
- failure-analysis
pretty_name: SEAS frozen-attacker control, contextual attack-specific refusals
seas-harden-ctx
Per-round trajectory + per-category taxonomy from an automated red-teaming co-evolution loop (attacker -> target -> harm-judge) on Qwen2.5 (7B attacker+judge, 3B target), seeded with JailbreakBench behaviors. This dataset is the frozen-attacker control, contextual attack-specific refusals arm: held-in ASR flat ~0.45 across 5 rounds - even with proper contextual refusals, hardening does not reduce ASR.
Study question: does adversarial co-evolution ignite an arms race, and does safety-hardening generalize? Well-powered answer at this scale: no to both. Full study + code: https://github.com/shipbehaves/self-evolving-adversarial-safety
defensive research / dual-use
This dataset contains counts, ASR curves, a redacted category taxonomy, and per-round LoRA adapters only. Raw attack strings and the target's unsafe completions are NEVER published - the pipeline strips all attack/response text before upload. No operational uplift.
configs
trajectory- per round: ASR held-in / held-out, over-refusal (benign control), judge drop-rate.taxonomy- per round x category: ASR and counts.adapters/round_*- per-round LoRA adapters.
ASR = attack success rate (fraction of responses a pointwise harm-judge labels unsafe). Held-out categories are never trained on (the generalization test).