← back to catalog · registered 2026-11-13 07:03

hUh63/Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M-GGUF

hUh63 9B GGUF second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/hUh63%2FQwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M-GGUF"
Response includes
  • classification m3
  • files 9
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
723
Likes
0
Model age
1w ago
created 2026-11-13
Downloads over time
Now 723 → from 0 ↑ 0%
0265530795 0 on Nov 13 723 on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh es fr ja ko ru
Quantizations
Q4_K
Tags
llama.cpp gguf mtp nextn speculative-decoding imatrix q4_k_m heretic abliterated uncensored conversational text-generation

Related

Total size
5.53 GB
Files
9
Quantizations
3
Registered
2026-11-13 07:03
Last updated on HF
2026-11-13 06:33

Files by quantization

Q4_K 1 file 5.53 GB
Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M.gguf 5.53 GB 4846e5a5 download
F16 1 file 876 MB
Qwen3.8-9B-Distill-uncensored-heretic-MTP-mmproj-f16.gguf 876 MB 355b88da download
Auxiliary files 7 files 629 KB
calibration.txt 600 KB 711fe1aa download
LICENSE 11.3 KB f938136e download
perplexity-corpus.txt 9.50 KB 10dcb200 download
README.md 6.51 KB 3a4e57ac download
.gitattributes 1.66 KB e92ae476 download
SHA256SUMS.txt 244 B 67a7998b download
tensor_types.txt 18.0 B ff96bc3c download

README current version from Hugging Face


license: apache-2.0
base_model:

  • petruhonk/Qwen3.8-9B-Distill-uncensored-heretic
    pipeline_tag: text-generation
    library_name: llama.cpp
    language:
  • en
  • zh
  • es
  • fr
  • ja
  • ko
  • ru
    tags:
  • gguf
  • llama.cpp
  • mtp
  • nextn
  • speculative-decoding
  • imatrix
  • q4_k_m
  • heretic
  • abliterated
  • uncensored
  • conversational

Qwen3.8-9B-Distill-uncensored-heretic — MTP — Q4_K_M GGUF

Q4_K_M quantization of
petruhonk/Qwen3.8-9B-Distill-uncensored-heretic,
with the built-in MTP (NextN) head kept in the same file and quantized at
Q8_0 instead of Q4_K so draft acceptance stays high.

  • Text trunk: Q4_K_M (general.file_type = 15, 5.06 BPW class, mixed Q4_K / Q6_K)
  • MTP block blk.32.*: Q8_0 for the 8 weight matrices, norms stay F32
  • 394 tensors, qwen35.block_count = 33, qwen35.nextn_predict_layers = 1
  • Quantized with a real activation importance matrix (--nextn included)
  • No separate draft model file needed: --spec-type draft-mtp reads blk.32.nextn.*
    from this GGUF
File Size Contents
Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M.gguf 5,939,570,176 B text trunk + MTP head
Qwen3.8-9B-Distill-uncensored-heretic-MTP-mmproj-f16.gguf 918,165,952 B vision projector (F16), for images/video

SHA256SUMS.txt holds the checksums; calibration.txt, perplexity-corpus.txt and
tensor_types.txt are the exact inputs used, so the quantized file is reproducible
byte-for-byte from the BF16 GGUF.

Why the MTP head is quantized differently

MTP is a pure draft layer: the trunk verifies every proposal and discards rejects, so
the head can never change the output distribution — it only decides how many tokens
each verification step accepts
. A 4-bit draft head drifts more from the trunk, which
shows up as a lower acceptance rate, which is exactly the throughput you were trying to
buy. Measured on this repo's own weights, 8 CPU cores, greedy, 6 prompts × 300 tokens:

MTP block precision Size Draft acceptance Mean accepted length Generation
Q8_0 (this repo) 5.94 GB 1216 / 1722 = 70.6 % 3.14 9.45 tok/s
Q4_K (default Q4_K_M) 5.83 GB 1213 / 1731 = 70.1 % 3.13 9.63 tok/s

Read that table honestly: on this machine the two are within noise of each other
(±0.5 pp acceptance, run-to-run spread 59–89 %), and Q8_0 costs 102 MiB (+1.8 %). The
Q8_0 choice is insurance, not a measured win at this scale — a single 9B-class MTP layer
is barely perturbed by 4-bit quantization, but on smaller draft heads and longer
generations the gap grows, and 102 MiB is cheap. If you need the smallest file, quantize
blk.32.* at Q4_K; nothing else changes.

Quality: how much did Q4_K_M cost?

Perplexity on a held-out multilingual corpus (perplexity-corpus.txt, 4 chunks × 512
ctx, unrelated to the imatrix text), same binary, same flags:

Model Perplexity
BF16 GGUF (conversion of the source weights) 5.3983 ± 0.45004
This Q4_K_M 5.4039 ± 0.44885

+0.10 % relative. That is a perplexity number, not a benchmark score — Q4_K_M is lossy,
and a 0.1 % PPL delta does not mean "no quality loss" on reasoning-heavy prompts. What
it does support is that the imatrix-driven recipe did not visibly break the model.

Usage

llama.cpp

llama-server -m Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M.gguf \
  -c 32768 -np 1 -t 8 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 3

--spec-type draft-mtp is required — without it the NextN layer is loaded but never
executed, and you get no speedup. Do not pass -md / a second model: the drafter
lives inside this file. --jinja enables the chat template's tool-calling path.

Images/video:

llama-mtmd-cli -m Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M.gguf \
  --mmproj Qwen3.8-9B-Distill-uncensored-heretic-MTP-mmproj-f16.gguf \
  --image photo.jpg -p "Describe the image."

Ollama

cat > Modelfile <<'MF'
FROM ./Qwen3.8-9B-Distill-uncensored-heretic-MTP-Q4_K_M.gguf
MF
ollama create qwen38-heretic-mtp -f Modelfile

Provenance and credits

Weights, training and the ablation are entirely the work of others; this repository
adds only GGUF conversion + imatrix quantization. Full chain:

  1. Qwen (Alibaba) — Qwen/Qwen3.5-9B, the base model family (Apache-2.0).
  2. Empero — empero-ai/Qwen3.8-9B-Distill,
    a full-parameter distillation of Qwen3.8 2.4T-A95B into the Qwen3.5-9B architecture
    (~70k curated teacher traces). The 15 mtp.* tensors originate here and are
    byte-for-byte identical in the repository below — I verified all 15 via HTTP
    Range requests (sha256 per tensor), not by trusting the README.
  3. petruhonk — petruhonk/Qwen3.8-9B-Distill-uncensored-heretic,
    the direct source of these weights: a Heretic
    ablation (60-trial coarse→mid→fine sweep) that reports 6/100 refusals at KL 0.0306 over
    the distillation. The ablation touched only attn.o_proj / mlp.down_proj in the 32
    main layers; the MTP block is untouched base weights, which is why the head still works.
  4. hUh63 — BF16 GGUF conversion with the head folded in as block 32, imatrix
    collection with --nextn, Q4_K_M quantization with the MTP block pinned to Q8_0,
    and the measurements above.

License: Apache-2.0, inherited from the upstream chain. If you redistribute, keep the
attribution above.

Limitations

  • This is a decensored / abliterated community model. Safety alignment has been
    deliberately weakened; quantization does not restore it. You are responsible for how
    you use it and for any local rules that apply.
  • Q4_K_M is a lossy format. Norms and biases stay F32 and the MTP block is Q8_0, which
    limits the damage, but 4-bit matrix weights are 4-bit matrix weights.
  • Acceptance rates above were measured on 8 CPU cores with greedy sampling and short
    generations. They vary with prompt style (code and structured math accept more than
    free-form prose), context length and sampling settings.
  • Perplexity deltas of this size cannot resolve differences on long-chain reasoning;
    benchmark the tasks you care about.
  • The text trunk is 32 layers; the vision tower lives in the mmproj file. Loading the
    GGUF without --mmproj gives you a text-only model.

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 Add model card, license, imatrix corpus and quantization recipe 334823a 6.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration