← back to catalog · registered 2026-11-13 09:03

KatterMobile/Qwen3.8-27B-abliterated-UD-GGUF

KatterMobile 27B GGUF
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/KatterMobile%2FQwen3.8-27B-abliterated-UD-GGUF"
Response includes
  • classification m8
  • files 3
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
173
Likes
0
Model age
1w ago
created 2026-11-13
Downloads over time
Now 173 → from 0 ↑ 0%
063127190 0 on Nov 13 173 on Nov 22 173 on Nov 19 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh
Quantizations
Q6_K
Tags
gguf llama.cpp abliterated uncensored qwen qwen3.8 hybrid-attention unsloth ud-quant text-generation en zh

Related

Total size
23.2 GB
Files
3
Quantizations
2
Registered
2026-11-13 09:03
Last updated on HF
2026-11-13 08:12

Files by quantization

Q6_K 1 file 23.2 GB
Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf 23.2 GB 81bfe401 download
Auxiliary files 2 files 9.56 KB
README.md 7.93 KB 49af3bc8 download
.gitattributes 1.62 KB c0c2bd42 download

README current version from Hugging Face


library_name: gguf
license: apache-2.0
language:

  • en
  • zh
    tags:
  • gguf
  • llama.cpp
  • abliterated
  • uncensored
  • qwen
  • qwen3.8
  • hybrid-attention
  • unsloth
  • ud-quant
  • text-generation
    base_model: unsloth/Qwen3.8-27B-GGUF
    base_model_relation: quantized
    pipeline_tag: text-generation

⚡ Qwen3.8-27B-abliterated (UD-Q6_K_L · Q8 ablated writers)

Uncensored hybrid-attention GGUF by KatterMobile (Kaden)

Hugging Face
Ko-fi
License


☕ A Personal Note from KatterMobile

"Hello! I spend a while figuiring out how to make these quantsizeatiosn, and if you notice any problems, feel free to open a PR or convo and be specific so I'm aware of the issue otherwise I wouldn't know."

☕ Support on Ko-fi: https://ko-fi.com/kattermobile
📬 Support & Inquiries: [email protected]


🌟 Model Overview

Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8 is an uncensored build of
unsloth/Qwen3.8-27B-GGUF UD-Q6_K_L
(24.18 GB), itself a dynamic quantization of Qwen/Qwen3.8-27B.

The refusal direction was removed in float32 inside the GGUF itself, then only the 68 edited
tensors were re-quantized — with Unsloth's own imatrix and the original per-tensor type recipe.
Every other tensor is byte-identical to the upstream UD file.

This is not a LoRA, not a fine-tune, and not a re-quantized safetensors dump: it is a surgical
weight edit of the exact artifact you run.

📊 At a Glance

Parameters 27.321 B (866 tensors)
File size / bpw 24.91 GB · 7.29 bpw
Architecture qwen35 — hybrid linear/full attention (48 SSM + 16 attention blocks) + 1 MTP block
Layers ablated 18 – 51 (34 blocks)
Tensors ablated 68 (ssm_out × 25, attn_output × 9, ffn_down × 34)
Context 262,144 tokens
Vision ❌ not included (text-only GGUF; upstream mmproj is untouched and separate)
MTP untouched (blk.64)
Quant Q6_K base with the 68 ablated tensors stored at Q8_0, Unsloth imatrix

🧠 How the Abliteration Was Done

  1. Reverse-engineered the GGUF. 65 blocks: 48 gated linear-attention (SSM) blocks
    (attn_qkv 5120→10240, attn_gate 5120→6144, ssm_out 6144→5120) and full attention every
    4th block (attn_q 5120→12288 with fused gate, attn_k/v 1024, attn_output 6144→5120).
  2. Extracted per-layer refusal directions with llama-cvector-generator from 20 paired
    harmful/harmless prompts. Each direction.N is the mean residual-stream difference at the
    output of block N−1.
  3. Orthogonalized the residual writers in float32:
    W ← W − s · d (dᵀW) / (dᵀd) with s = 1.35.
    This removes the direction from everything those matrices write into the residual stream.
  4. Re-quantized only the edited tensors with llama-quantize, the Unsloth imatrix
    (imatrix_unsloth.gguf) and an anchored per-tensor type recipe. Untouched tensors are copied
    with quantize = cur_type != new_type, i.e. bit-exact.
  5. Verified the surgery: 798/866 tensors byte-identical, and the projection along the refusal
    direction driven from ≈1.5 % to ≈0.55 % with the sign inverted (over-ablation), so
    quantization noise can no longer restore it.

Why the ablated tensors are Q8_0

A naive scale-1.0 edit re-quantized back to Q6_K measures like this:

matrix before after scale-1.0 + Q6_K
ffn_down 1.66 % 0.41 %
ssm_out 1.80 % 0.06 %

Quantization noise re-injects ~25 % of the removed direction. Storing the 68 edited tensors at
Q8_0 and over-ablating at 1.35 is what actually closes the gap — the same trick huihui uses for
its _K_L files.

🆚 How This Differs From Huihui's Release

huihui-ai Qwen3.8-27B-abliterated this release
source bf16 safetensors → own quants unsloth UD-Q6_K_L GGUF, edited in place
layer band 18–51 18–51 (same, verified on this arch)
direction single global direction per-layer directions (cosine between band edges and layer 30 is only 0.60–0.76)
targets ssm_out, attn_output, ffn_down same 68 tensors
quant of targets Q8_0 for _K_L Q8_0 for the same reason
other tensors untouched untouched, byte-identical
MTP / vision untouched untouched

📈 Measured Results

Smoke evaluation: 20 refusal-inducing prompts, temperature 0, thinking disabled, keyword-scored.
Perplexity: wikitext-2 test, 24 × 512 tokens.

model refusal markers hard refusals PPL
base UD-Q6_K_L 19/20 12/20 6.0851
this release 0/20 0/20 6.1286 (+0.71 %)

[!NOTE]
This is a 20-prompt smoke test, not a safety evaluation. Run a few hundred held-out prompts and
a capability benchmark before relying on it for anything serious.

📁 Files

file size notes
Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf 24.9 GB the model
quantization/…-tensor-types-Q8targets.txt 28 KB exact llama-quantize --tensor-type-file recipe
control-vector/cvec_mean.gguf 1.3 MB the 64 per-layer refusal directions used
scripts/abliterate_gguf.py 10 KB the GGUF weight-surgery tool
scripts/ABLITERATION-RUNBOOK.md 6 KB full reverse-engineering + reproduction notes

🚀 Quick Start (llama.cpp)

llama-cli -m Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf \
  -ngl 99 -c 8192 --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain why the sky is blue."
llama-server -m Qwen3.8-27B-abliterated-UD-Q6_K_L-Q8.gguf \
  -ngl 99 -c 32768 --port 8080 --reasoning-budget 0

--reasoning-budget 0 disables the thinking block; drop it (or use -1) for full reasoning.

Recommended sampling

parameter value
temperature 1.0
top_p 0.95
top_k 20
min_p 0.0
presence_penalty 0.0
repetition_penalty 1.0
context 262,144 (native)

🔁 Reproduce It Yourself

# 1. per-layer refusal directions (llama.cpp, no transformers)
llama-cvector-generator -m Qwen3.8-27B-UD-Q6_K_L.gguf -ngl 99 \
  --positive-file harmful.txt --negative-file harmless.txt \
  --method mean -o cvec_mean.gguf

# 2. float32 orthogonalization of the 68 residual writers
python3 abliterate_gguf.py --src Qwen3.8-27B-UD-Q6_K_L.gguf --dst patched_f16.gguf \
  --directions cvec_mean.gguf --layers 18-51 --scale 1.35 --out-type f16

# 3. re-quantize with the original imatrix + per-tensor recipe
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file recipe.txt patched_f16.gguf out.gguf Q6_K 20

⚠️ Usage Warnings

  • This model has had its refusal behaviour removed on purpose. It will produce content that
    the base model declines to produce. That is the point of the release.
  • Not suitable for public-facing, underage, or high-security deployments.
  • You are responsible for what you generate with it and for complying with local law.
  • No warranty of any kind; provided as-is under Apache-2.0.

☕ Support independent open-source AI

Ko-fi

Ko-fi: https://ko-fi.com/kattermobile · Email: [email protected]

Built and verified on a single NVIDIA GB10 Grace Blackwell workstation.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 better readme 3fe271c 7.9 KB
    Loading...
  2. 2026-11-13 Upload folder using huggingface_hub bc542be 8.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration