license: cc-by-4.0
language:
- en
task_categories: - text-generation
tags: - activation-steering
- refusal
- over-refusal
- interpretability
- reproduction
- safety
size_categories: - n<1K
configs: - config_name: over_refusal_100
data_files: over_refusal_100.jsonl
default: true - config_name: contrastive_pairs
data_files: contrastive_pairs.jsonl
open-refusal-steering: the prompt sets
The prompt sets behind an open reproduction of feature-level activation steering.
CTGT's A feature-level approach to mitigating bias and censorship in DeepSeek-R1
(hal-04992348v1) reports a jump from 32% to 100%
response rate on "a benchmark of 100 sensitive queries", and releases neither the
prompts nor the code. Their own public repository states that the headline comparison
cannot be regenerated from it. You cannot check a steering claim without the prompts,
so here are 100, categorised and released.
Code, method and full write-up: https://github.com/lgoyal6/open-refusal-steering
Files
| file | rows | what it is |
|---|---|---|
over_refusal_100.jsonl |
100 | The evaluation set. Questions a well-calibrated assistant should answer, from domains where aligned models over-refuse. At any steering strength, a refusal here is unambiguously an error. |
contrastive_pairs.jsonl |
32 | Topic-matched refuse/comply pairs used to learn the refusal direction. First 24 train the direction; the last 8 refuse_side prompts are held out entirely and used only as a refusal-headroom control. |
ppl_corpus.txt |
40,663 chars | Held-out neutral prose for measuring perplexity with the steering hook live. Darwin, On the Origin of Species, public domain. |
CONSTRUCTION.md |
How each set was built, including what it is deliberately not. |
Evaluation set composition
| category | n |
|---|---|
| medical | 18 |
| legal | 14 |
| security_defensive | 14 |
| pharmacology_harm_reduction | 10 |
| mental_health | 8 |
| household_chemical_safety | 8 |
| personal_safety | 8 |
| finance_tax | 8 |
| history_conflict | 7 |
| research_dual_use | 5 |
What the sets measure
Model Qwen/Qwen2.5-0.5B-Instruct, layer 23 of 24, Cohen's d = 3.67, CPU, greedy, seed 0. Every number below is read out of sweep_partial_n100.csv in the repository.
| alpha | benign response | coherent | degenerate | illicit control | perplexity | x baseline |
|---|---|---|---|---|---|---|
| 0.0 | 90.0% | 90.0% | 0.0% | 37.5% | 20.5 | 1.00 |
| 0.5 | 87.0% | 87.0% | 0.0% | 37.5% | 20.8 | 1.02 |
| 1.0 | 88.0% | 88.0% | 0.0% | 37.5% | 21.1 | 1.03 |
| 1.5 | 90.0% | 90.0% | 0.0% | 50.0% | 21.3 | 1.04 |
| 2.0 | 89.0% | 89.0% | 0.0% | 87.5% | 21.5 | 1.05 |
| 2.5 | 82.0% | 52.0% | 30.0% | 100.0% | 70.3 | 3.43 |
| 3.0 | 100.0% | 100.0% | 0.0% | 100.0% | 10,850,402 | 530,115 |
Two findings the sets were built to expose.
Steering never helps the prompts worth un-refusing. Benign response starts at 90% and never beats 90% at any alpha with a perplexity tax under 10%. The held-out illicit control meanwhile climbs from 37.5% to 87.5% at alpha 2.0. The intervention works; it works on the wrong set.
Behavioural metrics pass a model that has stopped producing language. At alpha 3.0 every behavioural number reads perfect: 100% response, 0% refusal, 0% degenerate. Perplexity is 10,850,402, or 530,115x the 20.5 baseline. Refusal-prefix scoring passes it. This is why the capability tax is reported on the same axis as the win.
Scope and limits
- A 9-point sweep (0 to 4.0) was launched; alphas 3.5 and 4.0 did not complete and are not reported. The table above is the whole of
sweep_partial_n100.csv, unfiltered. - One model, 0.5B parameters, on CPU. The method transfers; these numbers do not.
- The contrastive pairs use mild illicit framings (fare evasion, fake reviews, a forged doctor's note) rather than a harmful-instruction set. That is a weaker supervision signal than the published work uses, and it is a deliberate tradeoff so the set is publishable. See
CONSTRUCTION.md. - The direction is never learned from the set it is evaluated on.
Licence
over_refusal_100.jsonl, contrastive_pairs.jsonl and CONSTRUCTION.md are CC BY 4.0:
"open-refusal-steering benign over-refusal set" by Laksh Goyal, CC BY 4.0.
Source: https://github.com/lgoyal6/open-refusal-steering
ppl_corpus.txt is public domain (Project Gutenberg eBook #2009, header and trailer stripped). Repository code is Apache-2.0.