license: apache-2.0
base_model:
- Qwen/Qwen3Guard-Gen-4B
tags: - abliteration
- safety-guard
- guard-model
- qwen3guard
- safety-classifier
- activation-steering
library_name: transformers
pipeline_tag: text-generation
language: - en
Qwen3Guard-Gen-4B-Abliterated (guard-lane)
A safety-classifier abliteration of Qwen/Qwen3Guard-Gen-4B: the harmful-content
flagging behavior is removed while the model keeps scoring/prompts-and-responses.
This is not a chat-model refusal ablation. The target is the guard's verdict:
pristine flags 78/80 compliant harmful pairs; this edit flags 0/80 while
benign selectivity stays 80/80 Safe and the margin ranking survives
(AUC 0.975).
Method (short)
Direction from contrast at the decision step (response-moderation read), excised
from a single layer's o_proj + down_proj tensors (2 tensors, no lm_head),
unit-norm direction, geometric-alpha ladder bracketed to a clean flip.
- Edit: layer 32 at alpha = 6.0
- Recipe: steer-test first (asymmetric gate), profile only nominates. Verification
from the reloaded checkpoint.
Results
Pristine acceptance (instrument certification)
| Gate | Value |
|---|---|
| harmful flagged | 78/80 (2 reads Safe+Refusal-Yes, scoped out) |
| benign Safe | 80/80 |
| AUC (margin ranking) | 0.988 |
| parse/margin divergence | 0 |
Excise grid (held-out test split, 80+80)
| Layers | α | harmful flagged | harmful margin | benign Safe | AUC |
|---|---|---|---|---|---|
| L20 | 1.0 | 78/80 | +7.55 | 80/80 | 0.9875 |
| L22+26 | 1.0 | 78/80 | +6.32 | 80/80 | 0.9874 |
| L22+26 | 2.0 | 78/80 | +4.13 | 80/80 | 0.9855 |
| L22+26 | 4.0 | 47/80 | +0.03 | 80/80 | 0.9838 |
| L22+26 | 6.0 | 14/80 | −0.87 | 80/80 | 0.9497 |
| L32 | 1.0 | 78/80 | +5.93 | 80/80 | 0.9875 |
| L32 | 2.0 | 77/80 | +4.03 | 80/80 | 0.9875 |
| L32 | 4.0 | 61/80 | +0.22 | 80/80 | 0.9814 |
| L32 | 6.0 | 0/80 | −2.60 | 80/80 | 0.9755 |
| L32 | 8.0 | 0/80 | −3.92 | 80/80 | 0.9695 |
The adopted point is L32 α=6.0 (bolded): full flag suppression, full benign
selectivity, ranking intact at a margin HEADROOM cost of ~0.06 AUC vs the 1.0
edge; α=8 costs more AUC for no additional suppression. Reproduced three times
on the same recipe (identical gate numbers = checkpoint determinism).
Fresh-slice replication (unseen offset bank, 40+40, different subject model)
Both arms read unflagged (expected success shape); the margin axis is retained —
ablated-subject harmful margin −3.30 vs pristine −7.91 (4.6 points of retained
discrimination on unseen pairs with a different subject model).
Caveats (honest scope)
- Suppression is measured on the response-moderation read (the native
moderation mode). In-family evidence at 0.6B shows excise impact is
read-specific to a degree — the prompt-moderation read may retain partial
flagging of boundary items; treat any "the flag is gone" claim as
read-scoped. - Threshold-refit recovery: the ablated margins stay informative (they still
order harmful-vs-benign pairs above chance) — with a refitted threshold a
downstream user can partially re-flag; actual post-refit recovery at 4B was
not measured for this release and the 0.6B numbers (≈82% of bank at α 1.1)
do NOT transfer silently. - Text-adversarial robustness of the guard is unaffected by this analysis and
remains untouched. Known pristine-guard weaknesses (benign-flood recency
attacks) are properties of the base guard and should not be attributed to the edit. - Not for production moderation use. This is a research artifact for studying
the internal representation of guard verdicts at scale.
Method & tooling
Built with Fenrir (formerly
Absolver) — the abliteration toolkit that finds the fetters, bites, and documents
exactly what gave. Reference implementation for direction recovery (decision-step
contrast at the verdict token), transactional excision, steer-test-first
causality gating, E03-style held-out gates, and the margin-readout instruments
that keep the gate honest. The guard-lane protocol (pairs-not-prompts, AUC
ranking gates, read-scoped claims) is fully documented in the repo.
The gods bound the wolf — this time the fetter was a verdict, and the wolf ate it. 🐺
License
Apache-2.0, inherited from Qwen/Qwen3Guard-Gen-4B.