license: apache-2.0
base_model: Qwen/Qwen3.5-0.8B
tags:

• abliterated
• uncensored
• directional-ablation
• mechanistic-interpretability
• qwen3_5
• image-text-to-text
• conversational
  pipeline_tag: image-text-to-text
  library_name: transformers

Qwen3.5-0.8B — Refusal-Direction Abliterated

Directional ablation of the refusal direction in
Qwen/Qwen3.5-0.8B. No fine-tuning,
no gradient steps — only a rank-1 orthogonalisation of 24 weight matrices.

Method

1. A refusal direction u is estimated as the difference in mean residual-stream
   activations between 30 harmful and 50 harmless instruction prompts, taken at the
   last token of the chat-formatted prompt.

2. All 24 layers are swept and scored by the separation gap
   cos(harmful, u) − cos(harmless, u). The best layer is 23
   (gap 0.629; layers 0–2 give ~0.03–0.12, so the signal is strongly late).

3. u is projected out of the row space of every weight matrix in layers
   12–23 whose output space is the residual stream:

   W' = W − u (uᵀ W)
   

   Targets: mlp.down_proj (all 12 layers), plus self_attn.o_proj on the
   full-attention layers and linear_attn.out_proj on the linear-attention
   layers. Residual |Wu| after patching: 0.0000.

The vision tower (model.visual.*) and the MTP head (mtp.*) are untouched.

Results

Refusal rate on 20 harmful instruction prompts, greedy decoding, non-thinking mode:

Quality checks against the base model:

Known limitations

Abliteration removes the refusal behaviour; it does not add capability. On
this 0.8B model the compliant answers to harmful prompts are frequently
incoherent or factually wrong, because the base model is too small to produce
technically correct instructions. Ablation also measurably degrades factual
precision in ordinary answers — the base model is more reliable on questions
about specific real-world details. This model is best used where over-refusal
is the bigger problem than factual accuracy.

Usage

from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration

model = Qwen3_5ForConditionalGeneration.from_pretrained(
    "xdchinza/Qwen3.5-0.8B-abliterated", dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("xdchinza/Qwen3.5-0.8B-abliterated")

GGUF quantisation for llama.cpp / Ollama, including an i-matrix quant:
xdchinza/Qwen3.5-0.8B-abliterated-GGUF

ollama create qwen3.5-0.8b-abliterated -hf xdchinza/Qwen3.5-0.8B-abliterated-GGUF:Q4_K_M

Reproduction

Scripts used are documented in the repository README of the uploader. Summary
of parameters: 30 harmful / 50 harmless prompts, probe layer 23, strength 1.0,
layers 12–23.

License

Apache-2.0, inherited from Qwen/Qwen3.5-0.8B. Derivative of an Apache-2.0
model, redistributed under the same terms.