license: apache-2.0
base_model: Qwen/Qwen3-4B-Instruct-2507
base_model_relation: finetune
language:
- en
tags: - qwen3
- fp8
- compressed-tensors
- abliterated
- refusal-direction
- research
- interpretability
pipeline_tag: text-generation
library_name: transformers
Qwen3-4B-Instruct-2507 — Abliterated, FP8 (W8A8)
A research artifact: the "refusal direction" (Arditi et al. 2024,
arXiv:2406.11717) removed from
Qwen/Qwen3-4B-Instruct-2507,
then quantized to FP8. Its purpose is to measure the quality cost of
abliteration with paired statistics on held-out data — a number the ecosystem
of "uncensored" uploads reports for essentially no one.
If you want a safety-aligned assistant, use the base model. This is an
instrument for studying what abliteration does to a model, published with the
measurements that justify that framing.
What was done
Closed-form, no training:
- Last-token residual activations collected for 128 refused prompts (AdvBench)
vs. 128 harmless prompts (Alpaca), per layer. refusal_dir = normalize(mean(harmful) − mean(harmless)).- Candidate layers swept by measured held-out refusal rate; layer 24
(depth 0.67 of 36) won. - Every residual-writing matrix orthogonalized against that direction
(embed_tokens, per-layero_projandmlp.down_proj):W ← W − r̂(r̂ᵀW). - Re-quantized to FP8 (
FP8_DYNAMIC, per-channel weights + per-token dynamic
activations,lm_headexcluded) — identical recipe to the non-abliterated
FP8 sibling.
Pipeline and eval code: https://github.com/Subalzt/qwen3-4b-quantized-fp8-gptq
Measured cost (paired, vs. the non-abliterated FP8)
Harness validated — the non-abliterated FP8 reproduced PPL 10.0415 to the digit.
| metric | base FP8 | this model | Δ | paired test |
|---|---|---|---|---|
| Refusal (AdvBench held-out) | 100% | ~17% | −83 pp | intended effect |
| HumanEval pass@1 | 0.8659 | 0.8110 | −5.49 pp | McNemar p = 0.093 (not significant) |
| Perplexity (wikitext-2, ctx 2048) | 10.0415 | 11.9987 | +19.49% | paired t, p ≈ 2e-112 (145/145 chunks worse) |
The two capability metrics disagree, and that is the finding. A functional
code eval sees no significant loss; a dense per-token perplexity measure sees a
large, unambiguous degradation on every chunk. The +19.5% perplexity cost is
roughly 6× the +3.04% cost of 4-bit GPTQ quantization on the same model —
abliteration is a larger quality intervention than quantization itself.
Intended use
Research on safety-tuning mechanisms, refusal-direction interpretability,
quantization × behavioral-edit interaction, and robustness evaluation.
Limitations & responsible use
- Refusal behavior is substantially reduced by design. It is not safety-
aligned and will comply with many requests the base model declines. Deploy
only with your own safeguards and within applicable law and the Apache-2.0
terms inherited from the base model. - General language-modeling quality is measurably degraded (+19.5% PPL); do not
treat it as quality-equivalent to the base. - FP8 (compressed-tensors) runs under vLLM / transformers on Linux. For native
Windows use a GGUF build instead.
Run (vLLM)
vllm serve Subalzt/Qwen3-4B-Instruct-2507-abliterated-FP8 --max-model-len 8192
Citation
Arditi et al., Refusal in Language Models Is Mediated by a Single Direction,
2024. Base model © Qwen, Apache-2.0.