license: mit
task_categories:
- text-generation
language: - en
tags: - jailbreak
- adversarial-robustness
- circuit-breakers
- judge-disagreement
size_categories: - 10K<n<100K
RR (Circuit Breakers) attack completions with three-judge scores
This dataset bundles attack completions generated againstGraySwanAI/Llama-3-8B-Instruct-RR
(the "circuit breakers" defense), each scored by three independent judges:
local:strongreject(Lin et al., StrongREJECT classifier — most permissive)local:harmbench(HarmBench classifier — middle)local:gpt_oss(gpt-oss-safeguard-20b — strictest)
Headline finding: judges DISAGREE dramatically on certain attacks
Some attack families (notably inpainting and BoN) produce many StrongREJECT-flagged
"successes" that gpt-oss-safeguard does not confirm. These are likely judge false positives —
StrongREJECT scores the form of the response (looks-like-compliance) rather than the
content (actual harmful information delivered). Inspecting the SR-positive completions
shows tangential, off-topic, or generic public-health-style replies that any human reader
would not classify as a successful jailbreak.
| attack family | completions | SR ≥ 0.5 (ASR) | gpt-oss ≥ 0.5 (ASR) | both agree |
|---|---|---|---|---|
| BoN N=100 × 300 HarmBench prompts | 30,000 | 1,397 (4.7%) | 7 (0.0%) | 7 |
| Inpainting N=500 × 50 JBB prompts | 25,000 | 1,237 (4.9%) | 0 (0.0%) | 0 |
Per-prompt ASR (≥1 of N steps succeeds), for context: Inpainting → SR=100% / gpt-oss=0%.
Implication: when measuring jailbreak ASR against representation-engineering
defenses such as circuit breakers, single-judge ASR is unreliable. Triangulating
across judges with different biases (strict vs. lenient, content vs. form) is
essential to avoid false-positive ASR inflation.
Schema
One row per (behavior, attack_step, completion) triple. Each row has:
| field | type | description |
|---|---|---|
attack |
str | bon or inpainting |
attack_in_config |
str | attack identifier as configured at generation time |
model |
str | target model id (always GraySwanAI/Llama-3-8B-Instruct-RR) |
dataset |
str | source dataset (adv_behaviors for BoN, jbb_behaviors for inpainting — see notes below) |
behavior_index |
int | index into the original behavior list |
step_index |
int | which step of the multi-step attack |
completion_index |
int | within-step completion id (always 0 for BoN/inpainting) |
harmful_prompt |
str | the original harmful behavior prompt |
target_prefix |
str | a "Sure, here's how to..." style target prefix saved with the original conversation |
attacked_prompt |
str | the actual user-side prompt sent to the model after the attack transformation |
response |
str | the model's completion |
score_strongreject |
float | StrongREJECT score_with_jailbreak (probability ∈ [0, 1]) |
score_harmbench |
float | HarmBench score_with_jailbreak (binary 0/1) |
score_gpt_oss |
float | gpt-oss-safeguard score_with_jailbreak (binary 0/1) |
Files
completions_bon.jsonl— BoN completions onlycompletions_inpainting.jsonl— Inpainting completions onlycompletions_all.jsonl— concatenation of the two
Notes / caveats
- Inpainting was run on
jbb_behaviors(50 prompts) because the inpainting attack ships
with a hard-coded inpainting-prompt corpus keyed to JBB behavior IDs. BoN was run onadv_behaviors(HarmBench, 300 prompts). - The gpt-oss-safeguard scores were obtained by running the judge ONLY on completions
withscore_strongreject ≥ 0.5; for the rest the gpt-oss score is recorded as 0.0.
This is a compute optimization based on the (verified) finding that gpt-oss-safeguard
is strictly more conservative than StrongREJECT, so any completion SR rejects gpt-oss
would also reject. - Generation: greedy (temperature 0),
max_new_tokens=256,num_return_sequences=1.