license: apache-2.0
base_model: Qwen/Qwen3.5-2B
tags:
- abliterated
- refusal-blated
- safety-research
- qwen
- qwen3.5
Qwen3.5-2B Abliterated
Abliterated (refusal-direction removed) version of Qwen/Qwen3.5-2B.
Method
- Technique: Weight-level abliteration projecting refusal directions out of all 24 transformer layers
- Extraction: Simple mean-difference between refusal and compliance activations on system-prompt-paired prompts (no SVD whitening)
- Targets: Attention output projection (
o_proj/out_proj) + MLP down projection (mlp.down_proj) — every layer, both pathways - Alpha: 2.25
- Reference: Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (arXiv:2406.11717)
Performance
| Metric | Result |
|---|---|
| HarmBench refusal rate (320 prompts) | 0.6% (2/320) |
| Benign output quality | Preserved |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model = AutoModelForCausalLM.from_pretrained(
"PinoCookie/qwen3.5-2b-abliterated",
torch_dtype=torch.bfloat16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("PinoCookie/qwen3.5-2b-abliterated")
messages = [{"role": "user", "content": "Your prompt here"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Limitations
- Hallucinates on topics outside its knowledge, most likely because of the inherent size of the model
- General benchmark performance not evaluated; some capability regression possible
- Refusal removal is not guaranteed complete against all prompt formulations
Motivation
This model was created for safety and interpretability research. Understanding how refusal mechanisms can be removed informs the development of more robust
alignment techniques and defense strategies.