base_model: Qwen/Qwen3.8-27B
base_model_relation: adapter
library_name: peft
license: apache-2.0
tags:

• abliteration
• refusal-direction
• activation-steering
• control-vector
• glp
• lora
• qwen3

Qwen3.8-27B: abliteration as an adapter, not a checkpoint (GLP-49)

Two small files that remove refusal behaviour from Qwen/Qwen3.8-27B without
redistributing the model. The base checkpoint stays byte-identical.

For comparison, published abliterations of this same model are
30.9 GB and
20.6 GB.

Start with the LoRA. It works on stock tooling. The GGUF is here because on
architectures where a LoRA is impossible the projective format is the only route, and
this is the reference implementation of it.

Results

Measured on Qwen/Qwen3.8-27B at revision 1d4bf0f2ff60, greedy decoding, 300 new
tokens, three-way scored with a degeneracy guard.

Delivery, not refusal rate, so higher is better. The harmless control is held out of the
derivation entirely. The security suite contains no content resembling the derivation
contrast.

Over-refusal is 0 points on the held-out control. An earlier in-sample control read
−15.6, which is a stricter bound by construction: the direction was fitted to separate
the harmful set from that control, so it sits on the decision boundary.

Usage

from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3.8-27B"
model = AutoModelForImageTextToText.from_pretrained(base, dtype="bfloat16", device_map="cuda:0")
model = PeftModel.from_pretrained(model, "msuiche/Qwen3.8-27B-abliterated-cyber-GLP-49")
tok = AutoTokenizer.from_pretrained(base)

merge_and_unload() bakes it in if you want a merged checkpoint.

Do not scale the adapter

lora_alpha / r = 1.0, and α is already baked into lora_A. Raising it does not
strengthen removal. It reflects the component instead of removing it:

α = 1   h·d̂ → 0            the component is removed
α = 2   h·d̂ → −(h·d̂)       the component is REFLECTED

At α=2 this model refused 37.5 % of entirely harmless prompts (sourdough,
birdwatching, repotting a houseplant) with factual capability perfectly intact. That is
the same vector installing the behaviour rather than removing it.

If you want less than full strength, drop layers instead

This adapter covers layers 1 to 63. You do not have to use all of them, and coverage
is the safe dial where α is not. Measured on a 43-layer model, same direction and same
α, varying only how many layers were touched:

Partial coverage does less of the same thing. Raising α does something different and
dangerous. So to tune strength, subset the layers and leave α at 1.

That also makes targeted work possible. The direction is applied per layer, so you can
confine it to a depth range and ask what that range contributes, rather than treating
the model as one switch. Two cautions from doing this:

• Do not include layer 0. Steering it silenced this model completely: 96 outputs out
  of 96 empty, capability 0/12.
• Early layers matter more than their separation scores suggest. Dropping layers 10 to 17
  from a working span cost 9.4 points of delivery, even though a shuffled-label null test
  rates them weak.

To build a narrower adapter, re-run the derivation with the span you want. Editing this
file's tensor list is equivalent, since each layer's lora_A/lora_B pair is
independent.

One file, every quantisation

lora_A = −α·d̂ᵀW is computed once from the bf16 weights. Load it onto a quantised copy
of the same checkpoint and the error enters only through d̂ᵀ(W − W_q)x, which is the
quantisation noise projected onto a single direction and therefore suppressed by about
1/√5120:

At int4 the weights are twelve percent wrong and the intervention is still ninety-six
percent correct. One adapter should therefore cover the FP8, NVFP4, int8, int4 and GGUF
re-encodings of this checkpoint, rather than each needing its own abliterated upload.

This is measured on the arithmetic, not on behaviour. Delivery on a quantised base has
not been benchmarked.

What this is, and its limits

It is a rank-1 LoRA computed in closed form, not trained. No optimiser, no training
data:

B = d̂                  a direction in the residual stream
A = −α · d̂ᵀW           one matrix-vector product against the base weights

d̂ comes from a difference of means over 32+32 prompts at the last prompt token, with
the top 0.5 % of dimensions masked. This checkpoint has a massive-activation dimension
running 75 to 434 times the median, which otherwise dominates the estimate. Applied at
both residual writers across layers 1 to 63: 126 matrices, ‖ΔW‖/‖W‖ mean 0.0143.

Checkpoint-bound. lora_A contains W, so this file is valid for
Qwen/Qwen3.8-27B at revision 1d4bf0f2ff60 and its re-quantisations. It is not
portable to another model or another size. The method transfers to any dense model.
The file does not.

peft attaches an adapter to layer 0. target_modules is matched by suffix, so peft
hooks layer 0 as well, which this file deliberately excludes. It is inert only because
peft zero-initialises B, verified here as exactly zero contribution and zero weight
change after merge. That is a property of peft's defaults, not of this file, and it
matters because steering layer 0 silences the model.

Untested: vLLM and llama.cpp. Both should work, since vLLM permits LoRA on any
linear layer and llama.cpp has the qwen35 architecture, but neither has been loaded.

The GGUF needs a patched runtime. It declares glp.mode = project, meaning
h ← h − α(h·d̂)d̂. llama.cpp's built-in control vectors are additive, h ← h + s·d̂,
which is a different operation: it pushes every token along the axis instead of removing
the component, and it fails silently. A reader that does not understand glp.mode
must refuse the file rather than fall back to adding. Support lives in our llama.cpp fork (currently private).

Scoring caveat

The harness scores REFUSE / DEFLECT / COMPLY plus a degeneracy guard. It does not yet
separate a fourth state we found: the steered model rejecting a question's premise and
then answering in full. "I cannot provide instructions on how to make sourdough rise
improperly…", followed by 1,300 characters of correct instructions. Those score as
refusals, so over-refusal figures here are an upper bound.

Provenance

base            Qwen/Qwen3.8-27B @ 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0
contrast        32 harmful / 32 form-matched harmless, cross-domain
estimator       difference of means, 0.5% massive-activation masked, pooled
hook            post-layer residual, layers 1-63, alpha = 1
gguf sha256     dfdbcc06b30498224270e2f54d5784510b18aa185e0a50321e763373de83c805
lora sha256     32980b48abc4018255a1ee5a5c2fbd55ee110132482dd18c29b86e0b150334f7

A third party's direction, recovered from
published weights by SVD
and run through the same harness, scores 90.6 / 100 / 100, which is better than this
one on general refusal. That is recorded because it is true, and because the gap is the
contrast rather than the method: their prompt sets pushed through this pipeline reproduce
their result.

Intended use

Security research and evaluation. Measuring how robustly a capability is gated requires
being able to switch the gate off and observe what changes, and that work is only
possible on open-weight models.