license: other
base_model: Qwen/Qwen3.5-9B
tags:
- abliterated
- refusal-removal
- safety-research
- hybrid-architecture
- deltanet
language: - en
kaanhho/Qwen3.5-9B-abliterated-L19xL11
Qwen3.5-9B with surgical refusal removal via cross-layer direction transfer.
Method
This model was created by orthogonalizing Layer 11's output projection weights
against a refusal direction extracted from Layer 19 (a downstream attention layer).
The key finding: the optimal direction for removing refusal at a layer is NOT
the local mean-difference at that layer, but a direction from a downstream layer
where DeltaNet layers have disentangled the refusal signal. See our paper for details.
Technical Details
- Intervention site: Layer 11 (attention, 34% depth)
- Direction source: Layer 19 (attention, 61% depth) — DIM direction
- Modification:
self_attn.o_proj.weightandmlp.down_proj.weightorthogonalized - No hooks needed: modification is permanent in the weights
Results (Qwen3.5-9B)
| Benchmark | Original | This Model (expected) |
|---|---|---|
| SaladBench SR ASR | 8% | ~88% |
| JailbreakBench SR ASR | 6% | ~85% |
| TruthfulQA MC1 | 19% | ~22% |
| Perplexity | 6.16 | ~6.62 |
Citation
@article{hacihaliloglu2026refusal,
title={Refusal Geometry Across Heterogeneous Layer Types in Language Models},
author={Hac{\i}haliloglu, Kaan and Kek\"ulluoglu, Dilara},
year={2026}
}
Disclaimer
This model is released for safety research purposes only. It is intended to help
researchers understand and improve the robustness of safety mechanisms in hybrid
architectures. Do not use for harmful purposes.