license: apache-2.0
base_model: mrfakename/Qwen3.8-27B-abliterated
pipeline_tag: image-text-to-text
tags:
- gguf
- abliterated
- uncensored
Qwen3.8-27B (abliterated) GGUF
Huge thanks to Hugging Face for generously sponsoring the compute used to abliterate these models! This was done on HF Jobs, which makes it incredibly easy to run tasks such as these :)
Join the Discord: https://discord.com/invite/foss

GGUF quants of mrfakename/Qwen3.8-27B-abliterated, which is Qwen/Qwen3.8-27B with its refusal behaviour removed by directional ablation.
These are the unsloth/Qwen3.8-27B-GGUF quants with the edit applied in place. Only the edited tensors were dequantized, edited and requantized to their original type. Everything else is byte-identical.
| File | Size | Notes |
|---|---|---|
Qwen3.8-27B-abliterated-Q8_0.gguf |
27.1 GiB | Closest to the original weights. |
Qwen3.8-27B-abliterated-UD-Q6_K_XL.gguf |
23.6 GiB | |
Qwen3.8-27B-abliterated-UD-Q5_K_XL.gguf |
19.5 GiB | |
Qwen3.8-27B-abliterated-UD-Q4_K_XL.gguf |
16.4 GiB | Good default; fits a 24 GB card. |
Qwen3.8-27B-abliterated-UD-Q3_K_XL.gguf |
12.2 GiB | |
Qwen3.8-27B-abliterated-UD-Q2_K_XL.gguf |
9.3 GiB | Smallest. Loses a little of the edit (see below). |
What changed
- Refusal directions were measured separately for chat and think prompts at every layer, on the Q8_0 GGUF. The subspace candidates are the top singular vectors of all of those directions together.
- A multi-objective search (minimise refusals, keep the first-token KL on harmless prompts low) chose the subspace rank and per-layer strengths. It settled on a single direction (rank 1), at strength 0.8–1.06 across layers 13–63. The first 13 layers are left alone: editing them hurt quality a lot and did little for refusals.
- That direction was projected out of the residual-stream writes:
attn_outputon the 13 full-attention layers,ssm_outon the 38 linear-attention layers, andffn_downon all 51 of those layers. That is 102 tensors.
Nothing else was touched: layers 0–12, embeddings and the vision projector are byte-identical to the original quants.
Results (Q8_0 GGUF, greedy)
| refusals, chat (/100) | think mode, reasoning or answer refuses (/100) | KL, harmless first token | |
|---|---|---|---|
| original | 99 | 97 | 0 |
| abliterated | 0 | 1 | 0.110 |
The 100 prompts are held-out harmful requests. Chat replies were 384 tokens; think mode got 768 tokens. Where the reasoning had not finished by then, the reasoning text itself was judged. The KL figure includes a small noise floor from batched inference.
Does the edit survive quantization?
Mostly. Low-bit quants can round a small edit away, and a dense 27B has no shared expert held at high precision, so unsloth quantizes the edited ffn_down / ssm_out / attn_output tensors along with everything else. The patch log measures how much of the removed direction is left in each tensor after requantizing:
| File | edit lost, average | edit lost, worst tensor |
|---|---|---|
| Q8_0 | 0.1% | 0.1% |
| UD-Q6_K_XL | 0.3% | 1.9% |
| UD-Q5_K_XL | 0.9% | 2.6% |
| UD-Q4_K_XL | 1.5% | 6.9% |
| UD-Q3_K_XL | 2.7% | 12% |
| UD-Q2_K_XL | 6.5% | 33% (three IQ1_S ffn_down tensors) |
Q4 and up keep essentially all of it. UD-Q2_K_XL keeps most of it, but it was not separately evaluated and may refuse slightly more often. If that matters, use Q3 or higher. See patch-stats.txt for every tensor.
Running
Mainline llama.cpp runs these, e.g. llama-server -m Qwen3.8-27B-abliterated-UD-Q4_K_XL.gguf --mmproj mmproj-F16.gguf.
Files
Qwen3.8-27B-abliterated-*.gguf: the patched quants.mmproj-*.gguf: the unchanged vision projector.abliteration-lora.gguf: the edit as a LoRA, if you'd rather use another quant of the original:--lora abliteration-lora.gguf.abliteration-edit.npz: the subspace basis and per-layer strengths.patch-stats.txt: per-tensor patch log.
This model will comply with harmful requests. You are responsible for how you use it.