license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
tags:
- abliterated
- uncensored
- directional-ablation
- mechanistic-interpretability
- qwen3_5
- image-text-to-text
- conversational
pipeline_tag: image-text-to-text
library_name: transformers
Qwen3.5-0.8B — Refusal-Direction Abliterated
Directional ablation of the refusal direction inQwen/Qwen3.5-0.8B. No fine-tuning,
no gradient steps — only a rank-1 orthogonalisation of 24 weight matrices.
Method
A refusal direction
uis estimated as the difference in mean residual-stream
activations between 30 harmful and 50 harmless instruction prompts, taken at the
last token of the chat-formatted prompt.All 24 layers are swept and scored by the separation gap
cos(harmful, u) − cos(harmless, u). The best layer is 23
(gap 0.629; layers 0–2 give ~0.03–0.12, so the signal is strongly late).uis projected out of the row space of every weight matrix in layers
12–23 whose output space is the residual stream:W' = W − u (uᵀ W)Targets:
mlp.down_proj(all 12 layers), plusself_attn.o_projon the
full-attention layers andlinear_attn.out_projon the linear-attention
layers. Residual|Wu|after patching: 0.0000.
The vision tower (model.visual.*) and the MTP head (mtp.*) are untouched.
Results
Refusal rate on 20 harmful instruction prompts, greedy decoding, non-thinking mode:
| refusal rate | |
|---|---|
Base Qwen/Qwen3.5-0.8B |
18/20 (90%) |
| This model | 1/20 (5%) |
Quality checks against the base model:
| Metric | Base | This model |
|---|---|---|
| NLL, held-out neutral prose | 2.0713 | 2.0625 |
| Repeated 4-gram rate | 0.009 | 0.009 |
| Factual spot-checks (4) | 4/4 | 4/4 |
| Arithmetic (84*3/2) | correct | correct |
Known limitations
Abliteration removes the refusal behaviour; it does not add capability. On
this 0.8B model the compliant answers to harmful prompts are frequently
incoherent or factually wrong, because the base model is too small to produce
technically correct instructions. Ablation also measurably degrades factual
precision in ordinary answers — the base model is more reliable on questions
about specific real-world details. This model is best used where over-refusal
is the bigger problem than factual accuracy.
Usage
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
model = Qwen3_5ForConditionalGeneration.from_pretrained(
"xdchinza/Qwen3.5-0.8B-abliterated", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("xdchinza/Qwen3.5-0.8B-abliterated")
GGUF quantisation for llama.cpp / Ollama, including an i-matrix quant:xdchinza/Qwen3.5-0.8B-abliterated-GGUF
ollama create qwen3.5-0.8b-abliterated -hf xdchinza/Qwen3.5-0.8B-abliterated-GGUF:Q4_K_M
Reproduction
Scripts used are documented in the repository README of the uploader. Summary
of parameters: 30 harmful / 50 harmless prompts, probe layer 23, strength 1.0,
layers 12–23.
License
Apache-2.0, inherited from Qwen/Qwen3.5-0.8B. Derivative of an Apache-2.0
model, redistributed under the same terms.