license: apache-2.0
base_model: Qwen/Qwen3.8-27B
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- abliterated
- uncensored
- qwen3.8
- mtp
- vision
- multimodal
- not-for-all-audiences
Qwen3.8-27B - Abliterated (bf16 base)
Abliterated bf16 safetensors base of
Qwen/Qwen3.8-27B, with the
MTP head abliterated in-band and the vision tower preserved byte-identical.
Ready-to-run quants live in
RobinsonLabs/Qwen3.8-27B-abliterated-GGUF.
This repo is the full-precision master for further surgery (re-abliteration, LoRA merge,
fine-tune) and for rolling your own quants.
What is different about this one
Qwen3.8-27B picked up a lot of abliteration attention quickly. Two things here are, as far as
we can tell, not done elsewhere, and both are verified rather than asserted.
1. The MTP head is abliterated in-band. Qwen3.8-27B ships a multi-token-prediction head.
Most abliterations orthogonalize the 64-layer trunk and leave mtp.layers.0 untouched, because
the generic layer loop never reaches it. The draft head then keeps proposing refusal-prefix
tokens that the abliterated trunk rejects, and speculative acceptance collapses on exactly the
prompts abliteration exists to fix. Here the MTP block's two residual-write matrices
(self_attn.o_proj, mlp.down_proj) are orthogonalized with the same direction as the trunk.
MTP glue (mtp.fc, mtp.norm, mtp.pre_fc_norm_*) is deliberately untouched -- those are
norms and an input projection, not residual writers.
2. The vision tower is preserved byte-identical. All 333 model.visual.* tensors pass
through unmodified, verified by direct tensor diff (max delta 0.000000). An mmproj is
published in the GGUF repo so the vision half is actually usable, not just nominally intact.
Method
Single-direction weight orthogonalization (Arditi et al. style), applied to every matrix that
writes the residual stream.
| scope | tensor | count |
|---|---|---|
model.language_model.layers.* (64) |
mlp.down_proj |
64 |
linear_attn.out_proj (DeltaNet) |
48 | |
self_attn.o_proj (full-attn, interval 4) |
16 | |
mtp.layers.0 |
o_proj + down_proj |
2 |
model.language_model |
embed_tokens |
1 |
| edited | 131 | |
model.visual.* |
preserved byte-identical | 333 |
Coverage identity o_proj(16) + linear_out(48) == 64 == num_hidden_layers is enforced as a hard
gate before surgery writes a byte, which is what catches a partial match that would otherwise
produce a quietly half-abliterated model.
Direction selection. The refusal direction was captured twice, from two structurally
different chat-template renderings (one with enable_thinking=false, one with thinking on atreasoning_effort=xhigh, which injects an extra system block and shifts every token position).
The two agree at |cos| 0.96-0.99 across layers 18-45, peaking 0.9925 at layer 26, which is
the layer used. Two different prompt distributions converging on the same vector is evidence the
direction encodes refusal semantics rather than template formatting.
Attention-sink screen. Qwen3.8-27B's massive-activation dimension is 3994. It dominates
early layers (19-21% of direction energy at L1-L3) and orthogonalizing it out of every residual
writer produces a model that loads, runs, and emits garbage. Layer 26 carries only 0.06% of its
energy in dim 3994. Any re-derivation of this model should screen for it.
Measured behaviour
Base and abliterated probed in the same session, same harness, same 24 prompts, both at Q4_K_M:
| prompt set | base | abliterated |
|---|---|---|
| in-distribution (24, from the capture set) | 96% (23/24) | 8% (2/24) |
| held-out (40, disjoint split, overlap=0) | 100% (40/40) | 8% (3/40) |
| capability axis | abliterated |
|---|---|
| reasoning / code / math / factual / instruction-following | pass |
| creative / RP coherence | pass |
No capability regression on any axis: correct bat-and-ball, correct O(1)-space Fibonacci,
correct product rule, correctly rejects the "seasons are caused by distance" premise, and
returns exactly three comma-separated words when told to.
The held-out set is genuinely disjoint from the direction-capture set (416 train / 104 test,
overlap = 0), so the second row is not a reshuffle of prompts the direction was fitted on.
The refusal rate is the same 8% on both, which is the evidence that this generalizes rather than
having memorized its calibration data. The base refusing 40/40 on held-out prompts is also the
cleaner baseline, since it removes any suspicion that the capture set was cherry-picked for
prompts the base happened to refuse.
Quants
Ready-to-run GGUF quants are published at
RobinsonLabs/Qwen3.8-27B-abliterated-GGUF -- an eight-rung imatrix ladder cut from this repo's bf16
master, plus the f16 mmproj that restores the vision half.
| file | bits | size | bpw | fits |
|---|---|---|---|---|
Q8_0 |
8 | 29.05 GB | 8.51 | 2x24GB, or 32GB+ |
Q6_K |
6 | 22.43 GB | 6.57 | 24GB card, quality ceiling |
Q5_K_M |
5 | 19.54 GB | 5.72 | 24GB comfortable |
Q4_K_M |
4 | 16.84 GB | 4.93 | 24GB / 16GB with offload -- the volume rung |
IQ4_XS |
4 | 15.37 GB | 4.50 | 16GB card |
Q3_K_M |
3 | 13.59 GB | 3.98 | 16GB tight |
IQ3_XS |
3 | 12.26 GB | 3.59 | 12GB card |
IQ2_M |
2 | 10.30 GB | 3.02 | 10-12GB card -- quality-compromised, read the note |
Every rung is quantized from this master, so the ladder is a single lineage rather than a requant
chain. Download the mmproj alongside whichever rung you pick. The GGUF card documents one
calibration caveat worth reading if you re-quantize yourself: the imatrix does not cover the MTP
block.
Disclosure
This model is abliterated: the hard-refusal reflex on adult / creative content has been
reduced via single-direction weight orthogonalization. It will discuss material a stock instruct
model declines.
Harm guardrails are retained by design -- we ship at the ceiling where capability and
guardrails survive, not past it. Self-harm prompts still redirect to help (e.g. 988) rather than
comply, verified in probing. A residual fraction of requests are still refused outright; that is
the intended behaviour, not a shortfall in the ablation. Capability is preserved.
This is not a jailbreak-for-anything model and it is not intended to assist genuine wrongdoing.
Tagged not-for-all-audiences. Use responsibly, you are responsible for what you generate with
it. License inherited from the base model: Apache-2.0.
Provenance
Built by Robinson Labs with ModelForge, our
model-manufacturing system-of-record. Base pinned at commit 1d4bf0f2.