← back to catalog · registered 2026-11-13 05:03

hUh63/Qwen3.8-9B-Decensored-MTP-Q4_K_M-GGUF

hUh63 9B GGUF multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/hUh63%2FQwen3.8-9B-Decensored-MTP-Q4_K_M-GGUF"
Response includes
  • classification m8
  • files 21
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
524
Likes
1
Model age
1w ago
created 2026-11-12
Downloads over time
Now 524 → from 0 ↑ 0%
0192384576 0 on Nov 13 524 on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja ru es fr
Quantizations
Q4_K
Tags
gguf quantization q4_k_m imatrix llama.cpp mtp multi-token-prediction speculative-decoding qwen3_5 qwen3.8 image-text-to-text vision

Related

Total size
5.43 GB
Files
21
Quantizations
4
Registered
2026-11-13 05:03
Last updated on HF
2026-11-13 03:26

Files by quantization

Q4_K 1 file 5.43 GB
Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf 5.43 GB 804582d9 download
BF16 1 file 879 MB
mmproj-Qwen3.8-9B-Coder-BF16.gguf 879 MB 853698ce download
F16 1 file 876 MB
mmproj-F16.gguf 876 MB 448862e7 download
Auxiliary files 18 files 875 KB
calibration.txt 761 KB 7a070587 download
provenance_census.json 20.7 KB a8f921f6 download
verify_provenance.py 14.5 KB 56508c8a download
QUANTIZATION.md 12.8 KB 1a3b6ea0 download
LICENSE 11.1 KB d6456956 download
README.md 11.1 KB a31811ba download
PROVENANCE.md 11.0 KB f3eed686 download
heretic_ara_report.json 8.21 KB 1d7ab821 download
heretic_study.json 7.50 KB 33dc7d19 download
HERETIC_2_1_0_DEV0_METHOD.md 6.97 KB a5cfe89b download
upload_manifest.json 3.50 KB 614ed1ff download
config.toml 3.23 KB 256dba1c download
.gitattributes 1.68 KB aadeedaf download
spec_ab_report.json 781 B fbf095b6 download
online_verify.json 454 B ff3dc7f9 download
spec_mtp_evidence.txt 416 B 137f2c37 download
SHA256SUMS.txt 192 B 8a08cc67 download
tensor_types.txt 18.0 B 79d41c75 download

README current version from Hugging Face


library_name: gguf
license: apache-2.0
license_link: https://huggingface.co/empero-ai/Qwen3.8-9B-Distill/raw/main/README.md
base_model:

  • empero-ai/Qwen3.8-9B-Distill
  • Qwen/Qwen3.5-9B
    base_model_relation: quantized
    quantized_by: hUh63
    language:
  • en
  • zh
  • ko
  • ja
  • ru
  • es
  • fr
    pipeline_tag: image-text-to-text
    tags:
  • gguf
  • quantization
  • q4_k_m
  • imatrix
  • llama.cpp
  • mtp
  • multi-token-prediction
  • speculative-decoding
  • qwen3_5
  • qwen3.8
  • image-text-to-text
  • vision
  • multimodal
  • conversational
  • coder
  • code-generation
  • reasoning
  • chain-of-thought
  • tool-use
  • function-calling
  • agent
  • decensored
  • uncensored
  • abliterated
  • heretic
  • heretic-2.1.0.dev0
  • ara

Qwen3.8-9B — Decensored (Heretic v2.1.0.dev0) · Built-in MTP · Q4_K_M GGUF

A decensored + quantized derivative of
empero-ai/Qwen3.8-9B-Distill
by Empero (Apache-2.0): a full-parameter distillation of Qwen3.8 2.4T-A95B
into the Qwen3.5-9B architecture by the
Qwen team (Alibaba).

Decensoring used the genuine development snapshot
Heretic v2.1.0.dev0 by
Philipp Emanuel Weidmann (p-e-w) (commit 75a640b7eb, pyproject
version = "2.1.0.dev0"; stock ARA + Optuna TPE), with probe datasets by
mlabonne. The "keep the MTP layer at higher
precision to stabilize draft acceptance" idea follows the community write-up by
joeblowma. Quantized and published by hUh63 with
llama.cpp.

[!WARNING]
This build deliberately weakens the source model's safety alignment. It is a
decensoring experiment for research / local use. Compliance risk is yours.
Q4_K_M quantization is lossy. No standard benchmark (MMLU, GSM8K, …) was
run
— do not assume capability equals the aligned original. This is a
format conversion + decensoring + quantization only; all training
contributions belong to the authors credited below. 2.1.0.dev0 is a
development snapshot, not a stable Heretic release.

What was actually decensored

The request behind this build pointed at an Ollama tag. PROVENANCE.md takes that
tag apart tensor by tensor and reaches a conclusion worth stating up front:
its weights contain no fine-tune. All 184 of its unquantized (F32) tensors —
every norm, every SSM parameter — differ from empero-ai/Qwen3.8-9B-Distill only
by exact, invertible llama.cpp conversion steps: the 1+w RMSNorm fusion, the
grouped→tiled V-head permutation, and -exp(A_log). Zero tensors are unaccounted
for, and none shows a perturbed distribution. Its "coder" specialization is a
274-byte system prompt living in the manifest, not in the weights, and its vision
support is the ancestor's own vision tower.

That tag is also already a Q4_K_M GGUF, so re-quantizing it would be a second
lossy pass and Heretic cannot consume GGUF at all. Decensoring therefore ran on
the full-precision BF16 ancestor — which the census proves is the same weight
set, making it the only non-double-quantized source.

What is in this repo

File Description
Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf Text trunk, Q4_K_M, 33 blocks, built-in NextN/MTP layer (blk.32.*, kept at Q6_K)
mmproj-F16.gguf Vision projector, F16 (converted from the same BF16 source vision tower)
mmproj-Qwen3.8-9B-Coder-BF16.gguf The projector blob from the Ollama tag, unchanged (BF16+F32, 921,705,024 B, sha256 853698ce…); per PROVENANCE.md it is the ancestor's vision tower + merger, unmodified
calibration.txt Multilingual + math + code + reasoning corpus used for the importance matrix
tensor_types.txt Per-tensor precision override used by llama-quantize
config.toml Pinned Heretic v2.1.0.dev0 run configuration
heretic_study.json All Optuna trials, baselines, Pareto front, selected parameters
heretic_ara_report.json Changed-tensor diff stats + SHA256 of the 15 untouched MTP tensors
PROVENANCE.md Tensor-by-tensor proof that the Ollama target carries no fine-tune and is the BF16 ancestor in Q4_K_M
verify_provenance.py Re-derives every claim in PROVENANCE.md over HTTP Range requests, no full download
provenance_census.json Per-tensor classification of all 184 unquantized tensors (output of the script)
QUANTIZATION.md / HERETIC_2_1_0_DEV0_METHOD.md Measured, reproducible recipe and method
LICENSE Apache-2.0 (inherited from the base model lineage)

Why the MTP head is here

The BF16 ancestor ships 15 mtp.* tensors (mtp_num_hidden_layers=1).
transformers 5.20.2's Qwen3_5ForConditionalGeneration does not map them
into its state_dict, so Heretic's merged/exported weights omit them. We rebuild
the conversion input from the original index: only the matrices ARA actually
changed are replaced, and all 15 mtp.* (plus the vision tower, embeddings and
every norm) stay byte-identical to the base checkpoint (heretic_ara_report.json
has each SHA256). This also proves the decensor could not have touched the MTP head.

llama.cpp's convert_hf_to_gguf.py (_QwenMtpMixin) reads
mtp_num_hidden_layers = 1, raises block_count from 32 to 33, and maps
mtp.* into blk.32.nextn.* / blk.32.attn_* / blk.32.ffn_*. Note the
mapping of the two pre-fc norms is a cross assignment, not a name swap:
enorm ← mtp.pre_fc_norm_embedding, hnorm ← mtp.pre_fc_norm_hidden.
Getting it backwards produces a file that loads fine and drafts badly.
The result carries qwen35.nextn_predict_layers = 1, so speculative decoding uses
the built-in head — one file, no -md, no separate draft model.

Quantization (measured on the build machine)

8 CPU cores / 16 GiB RAM / no GPU / no swap / cgroup mem+memsw = 16 GiB
torch 2.11.0+cpu · transformers 5.20.2 · heretic-llm 2.1.0.dev0 · llama.cpp 93cf736a
  1. Decensor — the real Heretic v2.1.0.dev0 framework (source install at commit
    75a640b7eb), stock ARA + Optuna TPE, scaled down (few trials, small batches,
    short generations) to fit 8 CPU cores / 16 GiB. See config.toml and
    heretic_study.json.
  2. Rebuild conversion input with MTP restored byte-for-byte (build_gguf_src.py).
  3. BF16 GGUF (text, MTP inside) + F16 mmproj.
  4. imatrix on a real multilingual/math/code/reasoning corpus, with --nextn
    so the MTP layer gets importance statistics too.
  5. Q4_K_M with --tensor-type-file: ^blk\.32\..*=Q6_K lifts the whole MTP
    layer to Q6_K (norms stay F32), stabilizing draft acceptance for tens of MiB.
    general.file_type remains 15 (MOSTLY_Q4_K_M).

Full commands, pitfalls and static/runtime verification are in QUANTIZATION.md.

Usage

llama-cli \
  -m Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf \
  --mmproj mmproj-F16.gguf \
  --spec-type draft-mtp \
  -p "Explain speculative decoding." -n 128 -c 4096 -t 8

Do not pass a second -md draft model — the draft head is inside the file.
--spec-type draft-mtp is verify-then-accept, i.e. lossless: quantizing the
MTP head affects only the acceptance rate, never the output distribution.

The coder-leaning system prompt is not baked into the weights; pass it
yourself if you want that behavior:

--sys "You are Qwen3.8-9B, an expert programming assistant. When asked about your identity, model version, ..."

Measured results

Decensor (Heretic v2.1.0.dev0, 4 Optuna TPE trials, stock ARA + stock scorers)

Trial layer range Refusals KL divergence
baseline — 16/16 0
1 15–22 3/16 0.0516
2 (selected) 16–27 0/16 0.1544
3 5–30 0/16 0.1687
4 16–30 3/16 0.1123

trial_index=0 (Pareto sorted_index=0) was selected and is also the genuine
quality optimum this round: 0/16 refusals with low drift KL 0.1544 — unlike
earlier rounds where index 0 was 0/16 but KL ≈ 9.3 and had to be rejected.
Full study in heretic_study.json.

transformers 5.20.2 maps 760 keys, mtp.* = 0 → all 15 MTP tensors are
byte-identical to the BF16 ancestor (heretic_ara_report.json has each SHA256).
The source LFS blob was SHA256-verified against HF metadata.

Final build

Item Value
GGUF Qwen3.8-9B-Coder-Decensored-MTP-Q4_K_M.gguf
Size / sha256 5,828,751,552 B / 804582d970e7d044f2197f2d0c4f2ceb9154fe49df0289a802a32652d2fce538
BPW / file_type 5.06 / 15 (MOSTLY_Q4_K_M)
Tensors 442, block_count=33, nextn_predict_layers=1
Type distribution Q4_K:217 / F32:184 / Q6_K:41; blk.32.* = Q6_K:8, F32:7
mmproj (F16) mmproj-F16.gguf, 918,165,952 B, 334 tensors
mmproj (Ollama blob) mmproj-Qwen3.8-9B-Coder-BF16.gguf, 921,705,024 B, sha256 853698ce…, unchanged
BF16 intermediate 18,407,321,600 B, 442 tensors
imatrix --nextn, 16 chunks × 256 ctx, 10,671,040 B, corpus 779,147 B
ARA diff 22 matrices changed, all in layers 16–26; patch shard 1408 MiB; 15 mtp.* untouched
Draft A/B (8-core CPU) 8.3 → 10.8 tok/s (+30%); 68/79 (86.1%) and 66/82 (80.5%) accepted, mean accepted length 3.52/3.36; one file, no -md
Online check first 64 MiB of each published GGUF byte-identical to local (upload_manifest.json)

Honest limitations

  • Q4_K_M is lossy; imatrix + Q6_K MTP reduce but do not remove that loss.
  • Heretic v2.1.0.dev0 is a development snapshot; full defaults (100 trials, 400
    prompts, 100-token generations) were scaled down for 8 CPU cores — a small study.
  • 19.3 GiB > 16 GiB RAM: correctness relies on mmap page reclaim; it is slow.
  • Decensoring deliberately weakens safety alignment. Compliance risk is yours.
  • imatrix corpus is limited; no standard benchmark was run. Base-model-card scores
    belong to the full-precision aligned model, not this quant.
  • For a 9B model on pure CPU, speculative speedup can be modest (the draft itself
    costs compute); "lossless / single-file / no second model" still holds.
  • PROVENANCE.md proves what the requested Ollama artifact is; it says nothing
    about whoever published it, and this card makes no authorship claim about it.

Credits & license

  1. Qwen team (Alibaba) — Qwen3.5-9B base model family, Apache-2.0.
  2. Empero — empero-ai/Qwen3.8-9B-Distill, full-parameter distillation of
    Qwen3.8 2.4T-A95B into Qwen3.5-9B, Apache-2.0; direct source of all weights
    including the 15 MTP tensors and the vision tower.
  3. Unsloth — packaging of the Qwen3.5 vision projector that the Ollama
    mmproj blob carries (general.quantized_by: Unsloth in its GGUF header).
  4. p-e-w (Philipp Emanuel Weidmann) — Heretic v2.1.0.dev0 (ARA + Optuna TPE).
  5. mlabonne — harmless/harmful probe datasets.
  6. joeblowma — community经验:MTP 层保持高精度稳住投机解码接受率。
  7. ggml-org / llama.cpp — GGUF conversion, imatrix, quantization, runtime.
  8. hUh63 — decensor run, MTP-preserving rebuild, quantization and publishing.

License: Apache-2.0, inherited from the base model lineage. Quantization and
decensoring add no new restrictions.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-13 docs(provenance): rewrite from full 184-tensor census; drop LisyNeko authorsh... a1e63f4 11.1 KB
    Loading...
  2. 2026-11-12 Upload README.md with huggingface_hub 9107881 9.9 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration