license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
tags:
- qwen
- qwen2.5
- abliteration
- refusal-removal
- research
library_name: transformers
pipeline_tag: text-generation
Qwen2.5-7B-Instruct-Abliterated
A refusal-direction-removed variant of Qwen/Qwen2.5-7B-Instruct,
produced via the Heretic v1.2.0 abliteration framework.
⚠️ Research Use Only
This model is intended strictly for academic research, safety evaluation, and
red-teaming in controlled environments. It is not suitable for deployment
in production systems or any consumer-facing application.
The author assumes no liability for misuse.
Model Details
| Field | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-7B-Instruct |
| Processing Date | March 16, 2026 |
| Abliteration Tool | Heretic v1.2.0 |
| Selected Trial | Trial 415 (from 2200+ Optuna trials) |
| License | Apache 2.0 |
Methodology
Refusal directions were identified and suppressed via orthogonal projection acrossattn.o_proj and mlp.down_proj layers. Trial selection was performed using
Optuna with a composite objective balancing refusal-removal
rate and KL divergence from the base model distribution.
Evaluation Results
Evaluated on a set of 100 adversarial / edge-case prompts:
| Metric | Base Model | This Model |
|---|---|---|
| Refusal Rate | 99 / 100 (99%) | 3 / 100 (3%) |
| KL Divergence | — | 0.1049 |
The low KL divergence indicates that general language modeling capability
(Chinese/English fluency, instruction following, coding, mathematics, reasoning)
is largely preserved relative to the base model.
Intended Use
- Safety research: Studying refusal mechanisms and their robustness
- Red-teaming: Probing model behavior under adversarial prompts in a controlled lab setting
- Alignment research: Comparing behavior pre/post abliteration as a baseline
- Capability evaluation: Measuring the independence of refusal behavior from general capability
Limitations & Out-of-Scope Use
- This model has significantly reduced built-in safety guardrails.
It must not be used outside of isolated, controlled research environments. - Not intended for general-purpose chat, customer service, or any end-user deployment.
- Outputs should never be exposed to or acted upon in real-world contexts without
independent human review.
Quick Start
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "FangPingWu/Qwen2.5-7B-Instruct-Abliterated"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
messages = [{"role": "user", "content": "Your prompt here"}]
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=2048,
temperature=0.7,
top_p=0.9,
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
` ` `
---
## Citation
If you use this model in published research, please cite the original base model
and the Heretic abliteration tool.
---
## Related Work
- [Heretic: Abliteration framework](https://github.com/p-e-w/heretic)
- [Representation Engineering (Zou et al., 2023)](https://arxiv.org/abs/2310.01405)
- [Refusal in LLMs is mediated by a single direction (Arditi et al., 2024)](https://arxiv.org/abs/2406.11717)