← back to catalog · registered 2026-11-12 18:03

hUh63/Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M-GGUF

hUh63 9B GGUF multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/hUh63%2FQwen3.8-9B-Coder-Decensored-MTP-Q4_K_M-GGUF"
Response includes
  • classification m8
  • files 19
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
0
Likes
1
Model age
1w ago
created 2026-11-12
Downloads over time
Now 0 → from 0 ↑ 0%
0011 0 on Nov 12 0 on Nov 13 Nov
Nov 12 → Nov 13 · 2 snapshots · spans 1 day

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja ru es fr
Quantizations
Q4_K
Tags
gguf quantization q4_k_m imatrix llama.cpp mtp multi-token-prediction speculative-decoding qwen3_5 qwen3.8 image-text-to-text vision

Related

Total size
5.43 GB
Files
19
Quantizations
4
Registered
2026-11-12 18:03
Last updated on HF
2026-11-13 03:26

Files by quantization

Q4_K 1 file 5.43 GB
Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf 5.43 GB 804582d9 download
BF16 1 file 879 MB
mmproj-Qwen3.8-9B-Coder-BF16.gguf 879 MB 853698ce download
F16 1 file 876 MB
mmproj-F16.gguf 876 MB 448862e7 download
Auxiliary files 16 files 831 KB
calibration.txt 761 KB 7a070587 download
QUANTIZATION.md 12.8 KB 4a734cc7 download
LICENSE 11.1 KB d6456956 download
README.md 9.95 KB 1ebbf6fe download
heretic_ara_report.json 8.21 KB 1d7ab821 download
heretic_study.json 7.50 KB 33dc7d19 download
HERETIC_2_1_0_DEV0_METHOD.md 6.42 KB 11352847 download
upload_manifest.json 3.50 KB 614ed1ff download
PROVENANCE.md 3.49 KB e1593372 download
config.toml 3.23 KB 256dba1c download
.gitattributes 1.68 KB aadeedaf download
spec_ab_report.json 781 B fbf095b6 download
online_verify.json 454 B ff3dc7f9 download
spec_mtp_evidence.txt 416 B 137f2c37 download
SHA256SUMS.txt 192 B 8a08cc67 download
tensor_types.txt 18.0 B 79d41c75 download

README current version from Hugging Face


library_name: gguf
license: apache-2.0
license_link: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill/raw/main/README.md
base_model:

  • empero-ai/Qwen3.8-9B-Distill
  • Qwen/Qwen3.5-9B
    base_model_relation: finetune
    quantized_by: hUh63
    language:
  • en
  • zh
  • ko
  • ja
  • ru
  • es
  • fr
    pipeline_tag: image-text-to-text
    tags:
  • gguf
  • quantization
  • q4_k_m
  • imatrix
  • llama.cpp
  • mtp
  • multi-token-prediction
  • speculative-decoding
  • qwen3_5
  • qwen3.8
  • image-text-to-text
  • vision
  • multimodal
  • conversational
  • coder
  • code-generation
  • reasoning
  • chain-of-thought
  • tool-use
  • function-calling
  • agent
  • decensored
  • uncensored
  • abliterated
  • heretic
  • heretic-2.1.0.dev0
  • ara

Qwen3.8-9B-Coder — Decensored (Heretic v2.1.0.dev0) · Built-in MTP · Q4_K_M GGUF

Unofficial community derivative of LisyNeko/qwen3.8-9b-coder (Ollama).
The Ollama artifact is already a Q4_K_M GGUF; re-quantizing it would be a
second lossy pass, so decensoring runs on its full-precision BF16 ancestor
empero-ai/Qwen3.8-9B-Distill
by Empero (Apache-2.0), a full-parameter distillation of Qwen3.8 2.4T-A95B
into the Qwen3.5-9B architecture by
the Qwen team (Alibaba). PROVENANCE.md has byte-level proof that the
Ollama model's text trunk and vision tower are the same weights.

Decensoring here used the genuine development snapshot
Heretic v2.1.0.dev0 by
Philipp Emanuel Weidmann (p-e-w) (commit 75a640b7eb, pyproject
version = "2.1.0.dev0"; stock ARA + Optuna TPE), with probe datasets by
mlabonne. The "keep MTP layers at higher
precision to stabilize draft acceptance" idea follows the community write-up by
joeblowma. Quantized and published by hUh63 with
llama.cpp.

[!WARNING]
This build deliberately weakens the source model's safety alignment. It is a
decensoring experiment for research / local use. Compliance risk is yours.
Q4_K_M quantization is lossy. No standard benchmark (MMLU, GSM8K, …) was
run
— do not assume capability equals the aligned original. This is a
format conversion + decensoring + quantization only; all training
contributions belong to the authors credited above. 2.1.0.dev0 is a
development snapshot, not a stable Heretic release.

What is in this repo

File Description
Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf Text trunk, Q4_K_M, 33 blocks, built-in NextN/MTP layer (blk.32.*, kept at Q6_K)
mmproj-F16.gguf Vision projector, F16 (converted from the same BF16 source vision tower)
mmproj-Qwen3.8-9B-Coder-BF16.gguf LisyNeko's exact Ollama projector blob (BF16+F32, 921,705,024 B, sha256-verified), unchanged
calibration.txt Multilingual + math + code + reasoning corpus used for the importance matrix
tensor_types.txt Per-tensor precision override used by llama-quantize
config.toml Pinned Heretic v2.1.0.dev0 run configuration
heretic_study.json All Optuna trials, baselines, Pareto front, selected parameters
heretic_ara_report.json Changed-tensor diff stats + SHA256 of the 15 untouched MTP tensors
PROVENANCE.md Byte-level proof that the Ollama target and the BF16 ancestor share weights
QUANTIZATION.md / HERETIC_2_1_0_DEV0_METHOD.md Measured, reproducible recipe and method
LICENSE Apache-2.0 (inherited from the base model lineage)

Why the MTP head is here

The BF16 ancestor ships 15 mtp.* tensors (mtp_num_hidden_layers=1).
transformers 5.20.2's Qwen3_5ForConditionalGeneration does not map them
into its state_dict, so Heretic's merged/exported weights omit them. We rebuild
the conversion input from the original index: only the matrices ARA actually
changed are replaced, and all 15 mtp.* (plus the vision tower, embeddings and
every norm) stay byte-identical to the base checkpoint (heretic_ara_report.json
has each SHA256). This also proves the decensor could not have touched the MTP head.

llama.cpp's convert_hf_to_gguf.py (_QwenMtpMixin) reads
mtp_num_hidden_layers = 1, raises block_count from 32 to 33, and maps
mtp.* into blk.32.nextn.* / blk.32.attn_* / blk.32.ffn_*. The result
carries qwen35.nextn_predict_layers = 1, so speculative decoding uses the
built-in head — one file, no -md, no separate draft model.

Quantization (measured on the build machine)

8 CPU cores / 16 GiB RAM / no GPU / no swap / cgroup mem+memsw = 16 GiB
torch 2.11.0+cpu · transformers 5.20.2 · heretic-llm 2.1.0.dev0 · llama.cpp 93cf736a
  1. Decensor — the real Heretic v2.1.0.dev0 framework (source install at commit
    75a640b7eb), stock ARA + Optuna TPE, scaled down (few trials, small batches,
    short generations) to fit 8 CPU cores / 16 GiB. See config.toml and
    heretic_study.json.
  2. Rebuild conversion input with MTP restored byte-for-byte (build_gguf_src.py).
  3. BF16 GGUF (text, MTP inside) + F16 mmproj.
  4. imatrix on a real multilingual/math/code/reasoning corpus, with --nextn
    so the MTP layer gets importance statistics too.
  5. Q4_K_M with --tensor-type-file: ^blk\.32\..*=Q6_K lifts the whole MTP
    layer to Q6_K (norms stay F32), stabilizing draft acceptance for tens of MiB.
    general.file_type remains 15 (MOSTLY_Q4_K_M).

Full commands, pitfalls and static/runtime verification are in QUANTIZATION.md.

Usage

llama-cli \
  -m Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf \
  --mmproj mmproj-F16.gguf \
  --spec-type draft-mtp \
  -p "Explain speculative decoding." -n 128 -c 4096 -t 8

Do not pass a second -md draft model — the draft head is inside the file.
--spec-type draft-mtp is verify-then-accept, i.e. lossless: quantizing the
MTP head affects only the acceptance rate, never the output distribution.

The Ollama "coder" system prompt (You are Qwen3.8-9B, an expert programming assistant…) is not baked into the weights; pass it yourself if you want the
same behavior:

--sys "You are Qwen3.8-9B, an expert programming assistant. When asked about your identity, model version, ..."

Measured results

Decensor (Heretic v2.1.0.dev0, 4 Optuna TPE trials, stock ARA + stock scorers)

Trial layer range Refusals KL divergence
baseline — 16/16 0
1 15–22 3/16 0.0516
2 (selected) 16–27 0/16 0.1544
3 5–30 0/16 0.1687
4 16–30 3/16 0.1123

trial_index=0 (Pareto sorted_index=0) was selected and is also the genuine
quality optimum this round: 0/16 refusals with low drift KL 0.1544 — unlike
earlier rounds where index 0 was 0/16 but KL ≈ 9.3 and had to be rejected.
Full study in heretic_study.json.

transformers 5.20.2 maps 760 keys, mtp.* = 0 → all 15 MTP tensors are
byte-identical to the BF16 ancestor (heretic_ara_report.json has each SHA256).
The source LFS blob was SHA256-verified against HF metadata.

Final build

Item Value
GGUF Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf
Size / sha256 5,828,751,552 B / 804582d970e7d044f2197f2d0c4f2ceb9154fe49df0289a802a32652d2fce538
BPW / file_type 5.06 / 15 (MOSTLY_Q4_K_M)
Tensors 442, block_count=33, nextn_predict_layers=1
Type distribution Q4_K:217 / F32:184 / Q6_K:41; blk.32.* = Q6_K:8, F32:7
mmproj (F16) mmproj-F16.gguf, 918,165,952 B, 334 tensors
mmproj (author BF16) mmproj-Qwen3.8-9B-Coder-BF16.gguf, 921,705,024 B, sha256 853698ce… (LisyNeko's exact Ollama blob)
BF16 intermediate 18,407,321,600 B, 442 tensors
imatrix --nextn, 16 chunks × 256 ctx, 10,671,040 B, corpus 779,147 B
ARA diff 22 matrices changed, all in layers 16–26; patch shard 1408 MiB; 15 mtp.* untouched
Draft A/B (8-core CPU) 8.3 → 10.8 tok/s (+30%); 68/79 (86.1%) and 66/82 (80.5%) accepted, mean accepted length 3.52/3.36; one file, no -md
Online check first 64 MiB of each published GGUF byte-identical to local (upload_manifest.json)

Honest limitations

  • Q4_K_M is lossy; imatrix + Q6_K MTP reduce but do not remove that loss.
  • Heretic v2.1.0.dev0 is a development snapshot; full defaults (100 trials, 400
    prompts, 100-token generations) were scaled down for 8 CPU cores — a small study.
  • 19.3 GiB > 16 GiB RAM: correctness relies on mmap page reclaim; it is slow.
  • Decensoring deliberately weakens safety alignment. Compliance risk is yours.
  • imatrix corpus is limited; no standard benchmark was run. Base-model-card scores
    belong to the full-precision aligned model, not this quant.
  • For a 9B model on pure CPU, speculative speedup can be modest (the draft itself
    costs compute); "lossless / single-file / no second model" still holds.

Credits & license

  1. Qwen team (Alibaba) — Qwen3.5-9B base model family, Apache-2.0.
  2. Empero — empero-ai/Qwen3.8-9B-Distill, full-parameter distillation of
    Qwen3.8 2.4T-A95B into Qwen3.5-9B, Apache-2.0; direct source of all weights
    including the 15 MTP tensors and the vision tower.
  3. LisyNeko — qwen3.8-9b-coder on Ollama: coder system prompt/template +
    Qwen3.5 vision projector packaging; the exact BF16 projector blob shipped here
    is their artifact (general.quantized_by: Unsloth retained).
  4. Unsloth — Qwen3.5 vision projector packaging referenced by the Ollama mmproj.
  5. p-e-w (Philipp Emanuel Weidmann) — Heretic v2.1.0.dev0 (ARA + Optuna TPE).
  6. mlabonne — harmless/harmful probe datasets.
  7. joeblowma — community经验:MTP 层保持高精度稳住投机解码接受率。
  8. ggml-org / llama.cpp — GGUF conversion, imatrix, quantization, runtime.
  9. hUh63 — decensor run, MTP-preserving rebuild, quantization and publishing.

License: Apache-2.0, inherited from the base model lineage. Quantization and
decensoring add no new restrictions.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 docs(provenance): rewrite from full 184-tensor census; drop LisyNeko authorsh... a1e63f4 11.1 KB
    Loading...
  2. 2026-11-12 Upload README.md with huggingface_hub 9107881 9.9 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration