← back to catalog · registered 2026-11-14 18:03

hUh63/Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M-GGUF

hUh63 4B GGUF multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/hUh63%2FQwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M-GGUF"
Response includes
  • classification m8
  • files 33
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
138
Likes
0
Model age
1w ago
created 2026-11-14
Downloads over time
Now 138 → from 0 ↑ 0%
051101152 0 on Nov 14 138 on Nov 22 Nov
Nov 14 → Nov 22 · 9 snapshots · spans 8 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja ru es fr
Quantizations
Q4_K
Tags
gguf llama.cpp quantization q4_k_m imatrix mtp multi-token-prediction speculative-decoding qwen3_5 qwen3.8 image-text-to-text vision

Related

Total size
2.67 GB
Files
33
Quantizations
3
Registered
2026-11-14 18:03
Last updated on HF
2026-11-14 18:13

Files by quantization

Q4_K 1 file 2.67 GB
Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf 2.67 GB ebd5efb2 download
F16 1 file 641 MB
Qwen3.8-4B-Empero-FullStack-Decensored-MTP-mmproj-f16.gguf 641 MB 4bba1886 download
Auxiliary files 31 files 721 KB
calibration.txt 586 KB 770ae3d0 download
baseline.json 31.7 KB 08722534 download
search_lam.py 17.2 KB 76880a3a download
make_calibration.py 15.6 KB 87eaf043 download
README.md 12.2 KB 8219dbad download
METHOD.md 8.12 KB 54025d96 download
check_gguf.py 7.02 KB 6d8fd27e download
build_gguf_src.py 7.01 KB b77aa348 download
make_ppl_corpus.py 6.75 KB 9436ac8a download
heretic_ara_report.json 6.38 KB 2ffb3ea8 download
perplexity-corpus.txt 5.93 KB 7cce3a4b download
quantize.sh 4.79 KB 2c9882ee download
config.toml 3.45 KB 2d73fd6e download
seeds.json 2.14 KB f65d14fd download
.gitattributes 1.67 KB 008730fc download
LICENSE 968 B 7111689e download
summary.json 878 B 04f60e61 download
SHA256SUMS.txt 590 B 6a849d79 download
selected.json 489 B 428bf57f download
tensor_types_report.json 385 B 138598d0 download
tensor_types_report_default.json 385 B 6bccb4d4 download
tensor_types_report_mtpQ6K.json 376 B 33653b92 download
points.jsonl 282 B c71f27d6 download
curve-A_preserve082.json 250 B 26b4cfd5 download
ppl.json 241 B 4c1a9f1d download
calibration_manifest.json 231 B 90b3beba download
spec_default.json 228 B 96ed12fe download
spec_mtpQ6K.json 227 B 5ee4d361 download
spec_mtpQ8K.json 226 B 1b947015 download
pareto.json 117 B 2f830f7d download
tensor_types_q8_0.txt 18.0 B ff96bc3c download

README current version from Hugging Face


library_name: gguf
license: apache-2.0
license_link: https://huggingface.co/iBotIA/Qwen3.8-4B-Empero-AI-FullStack/raw/main/README.md
base_model:

  • iBotIA/Qwen3.8-4B-Empero-AI-FullStack
  • empero-ai/Qwen3.8-4B-Distill
  • Qwen/Qwen3.5-4B
    base_model_relation: quantized
    quantized_by: hUh63
    language:
  • en
  • zh
  • ko
  • ja
  • ru
  • es
  • fr
    pipeline_tag: image-text-to-text
    tags:
  • gguf
  • llama.cpp
  • quantization
  • q4_k_m
  • imatrix
  • mtp
  • multi-token-prediction
  • speculative-decoding
  • qwen3_5
  • qwen3.8
  • image-text-to-text
  • vision
  • full-stack
  • typescript
  • react
  • nestjs
  • flutter
  • decensored
  • uncensored
  • abliterated
  • heretic
  • heretic-2.1.0.dev0
  • ara

Qwen3.8-4B-Empero-FullStack — Decensored · built-in MTP · Q4_K_M GGUF

GGUF quantizations of iBotIA/Qwen3.8-4B-Empero-AI-FullStack,
decensored with Heretic v2.1.0.dev0 and converted with llama.cpp.

The upstream model is a full-stack engineering fine-tune (279k curated segments: React Router v8,
NestJS, Prisma / Drizzle, TypeScript strict mode, Tailwind + shadcn, Flutter) of
empero-ai/Qwen3.8-4B-Distill — a
full-parameter distillation of the Qwen3.8 2.4T-A95B teacher into the
Qwen3.5-4B architecture. Text weights and the MTP
head come from that lineage; the vision tower is the Qwen3.5 tower carried along with it.

[!WARNING]
This build deliberately weakens the source model's safety alignment. Research and local
use only — compliance risk is yours. Q4_K_M is lossy. No standard benchmark suite (MMLU,
GSM8K, HumanEval, …) was run on this derivative. 2.1.0.dev0 is a Heretic development
snapshot, not a stable release.

Files

File Quant Size Notes
Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf Q4_K_M, MTP layer Q8_0 2,869,418,944 B recommended; ~5.29 BPW
Qwen3.8-4B-Empero-FullStack-Decensored-MTP-mmproj-f16.gguf F16 vision projector 672,422,944 B required for image input

Two comparison builds were produced in the same run and are documented here rather than
shipped, to keep the repo small: Q4_K_M with the MTP layer at Q6_K (2,840,214,464 B B,
5.23 BPW) and plain Q4_K_M with no per-tensor override (2,815,884,224 B B).
Their acceptance-rate measurements are in spec_*.json.

SHA256:

ebd5efb2221b07d702cdb7ad351e25cb8ae9810ab2b25773bbdcc21719272b4e  Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf
4bba18869b9f61cdbd31d0bbe047994c343a82af8ac2e5371f3d98f5ead8c220  Qwen3.8-4B-Empero-FullStack-Decensored-MTP-mmproj-f16.gguf

What "MTP layer at high precision" means here

The source ships 15 mtp.* tensors (mtp_num_hidden_layers = 1) — a multi-token-prediction
draft head inside the same network
. convert_hf_to_gguf.py maps them onto block 32, so the
GGUF has block_count = 33 and nextn_predict_layers = 1.

Because speculative decoding is verify-then-accept, the draft head can never change the
output distribution — it only decides how many tokens per verification step get accepted.
Quantizing it therefore shows up as an acceptance-rate effect, not a perplexity effect.
That is exactly why it gets a precision override here:

Item Value
GGUF Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf
Tensors / blocks 393 / 33 (nextn_predict_layers = 1)
Type distribution F32:184 / Q4_K:168 / Q6_K:33 / Q8_0:8
blk.32.* (the MTP layer) Q8_0:8 / F32:7
BPW 5.29

Measured acceptance rate, 12 prompts × 300 tokens each, llama-server --spec-type draft-mtp,
identical sampling and identical trunk weights across the three columns:

MTP 层精度(其余同为 Q4_K_M) 池化接受率 各 prompt 平均接受率 平均接受长度
Q8_0(本仓库) 68.75% 69.2%(57.8–76.75%) 3.07
Q6_K(备选档) 67.97% 68.6%(56.16–76.75%) 3.05
默认档(未覆盖,Q4_K/Q6_K 混合) 67.63% 68.4%(55.06–77.98%) 3.04

Read the spread, not the ranking: single-run acceptance rates on this machine move by tens of
percentage points between runs, so a one-digit difference between columns is noise. The
numbers above are reported with their min/max so you can judge that yourself.

Quantization quality

Perplexity with llama-perplexity -c 512 --chunks 4 on a held-out corpus written for this
run (disjoint from the imatrix calibration text):

Model PPL Δ vs BF16
BF16 GGUF (33 blocks, built-in MTP) 10.0916 ± 1.14470 —
Q4_K_M, MTP Q8_0 (this repo) 10.3287 ± 1.17320 +2.35%
Q4_K_M, MTP Q6_K 10.3287 ± 1.17320 +2.35%
Q4_K_M, no MTP override (control) 10.3287 ± 1.17320 +2.35%

The three Q4 variants land within a hair of each other because perplexity never executes the
draft head
— it is a draft-only layer. Do not read the PPL table as evidence about MTP
precision; read the acceptance table above.

Decensoring — what was actually run

Heretic v2.1.0.dev0 (p-e-w/heretic @ 75a640b7eb, stock ARA modifier + stock
KeywordRate / KLDivergence scorers, framework source unmodified), driven by
search_lam.py from this repo. Machine: 24 cores / 48 GiB CPU, no GPU.

Protocol — this is the published one, applied identically to the base model and to every
candidate:

  • refusals: mlabonne/harmful_behaviors test[:100], greedy, 64 new tokens
  • drift: KL divergence of first-token distributions on mlabonne/harmless_alpaca test[:100]
    against the unmodified BF16 source
  • refusal detection: Heretic's own keyword markers, so a compliant answer that opens with
    ⚠️ Disclaimer counts as a refusal. The drop from the base rate is the signal, not the exact
    denominator.

Method: ARA (Arbitrary-Rank Ablation) direction + scalar magnitude λ. ARA produces the
ablation direction as LoRA adapters with lora_alpha = lora_rank = 16, so the deployed scaling
is alpha/r = 1 and lora_B is exactly the task vector's magnitude. For each of
3 directions this run fitted the direction once, then swept
W_eff(λ) = W_base + λ·(B@A), bounded λ by bisection at the KL budget, and scored only on the
100-prompt protocol. A deterministic smoke test (KL(λ=0) ≈ 0) guards against a mis-scaled
adapter.

方向 λ 曲线(λ → KL)
A_preserve082 1.00 → 0.0145, 1.40 → 0.0246, 1.80 → 0.0364, 2.20 → 0.0494, 2.60 → 0.0677

Results

Model Refusals KL vs base
iBotIA/Qwen3.8-4B-Empero-AI-FullStack (BF16 source) 97/100 0 (by definition)
this build (direction A_preserve082, λ = 1.6362) 14/100 0.0308
target ≤ 15/100 ≤ 0.04

Verdict: 目标达成:14/100 拒答、KL 0.0308,两个约束同时满足。

# 方向 λ 拒答 KL 判定
200 A_preserve082 1.6362 14/100 0.0308 HIT
201 A_preserve082 1.925 14/100 0.0399 HIT

共 2 个 100 条全量真评点。KL(λ) 曲线见仓库里的 curve-*.json(每个方向一份),全部原始点见 points.jsonl,没有丢弃任何点。

Pareto front (non-dominated, refusals × KL):

  拒答 14/100  KL=0.0308  (点 200, 方向 A_preserve082, λ=1.6362)

Prompt format

The chat template is inherited verbatim from the source. System message first, add_generation_prompt
opens a <think> block, so every answer starts with reasoning — give it room
(max_new_tokens ≥ 512 for anything non-trivial).

Upstream-recommended sampling for this distillation family: temperature=0.6, top_p=0.95,
top_k=20. Greedy decoding (temperature 0) is explicitly discouraged by the author of the
upstream model — reasoning models in this class fall into repetition loops.

Usage

llama.cpp (built after the MTP-supporting commit; --spec-type draft-mtp is required):

llama-cli -m Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf --spec-type draft-mtp \
  -p "Write a NestJS guard that rejects requests without a valid JWT." \
  -n 512 -c 8192 -t 8

Do not pass a second model with -md — the draft head lives inside this file. Without
--spec-type draft-mtp the head is loaded but never executed, which is slower than not using
speculation at all.

Vision:

llama-mtmd-cli -m Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf --mmproj Qwen3.8-4B-Empero-FullStack-Decensored-MTP-mmproj-f16.gguf \
  --image screenshot.png -p "Turn this UI mockup into a Tailwind + shadcn component."

llama-server (acceptance rate is only printed by the server's per-slot timing output):

llama-server -m Qwen3.8-4B-Empero-FullStack-Decensored-MTP-Q4_K_M.gguf --spec-type draft-mtp -c 8192 -np 1 --host 127.0.0.1 --port 8080

How this file was produced

Full runbook, scripts and every raw measurement are in the CNB repo
OpenOmniBot-omniinfer/Test under
docs/quantization/qwen38-4b-empero-fullstack/ — the method is public, the weights are not
mirrored there.

  1. Download the 6 BF16 shards (9,319,820,368 B, 738 tensors, 15 mtp.*),
    each verified against its Hugging Face LFS SHA256.
  2. Fit ARA directions, sweep λ, score 100 prompts per candidate (2 full points, 2 total records, 0.7 h).
  3. Rebuild the HF conversion input from the original index: only the 14 matrices
    ARA actually changed are written to a patch shard that sorts last; all 15 mtp.* tensors, the
    whole vision tower, embeddings and every norm stay byte-identical to the source checkpoint
    (heretic_ara_report.json carries the per-tensor SHA256 of the MTP head, patch shard
    505.0 MiB).
  4. convert_hf_to_gguf.py --outtype bf16 → BF16 GGUF with MTP as block 32; --mmproj --outtype f16
    for the projector.
  5. llama-imatrix --nextn over a ~500 KiB calibration corpus weighted toward this model's actual
    domain (60% TypeScript/TSX + Flutter + SQL, 20% multilingual prose, 20% math/reasoning).
    --nextn is required for the MTP layer to receive importance statistics at all.
  6. Three llama-quantize passes (Q8_0 / Q6_K / no override on blk.32.*), header assertions per
    build, PPL on a disjoint corpus, acceptance-rate A/B.

Honest limitations

  • The refusal target is a keyword count, not a judgment. 目标达成。
  • Q4_K_M is lossy: +2.35% perplexity relative to BF16 on this corpus.
  • Acceptance-rate A/B is 12 prompts per column. That is enough to see the order of magnitude,
    not enough to resolve a 2-point difference — and per-run variance here is large.
  • 2.1.0.dev0 is a development snapshot of Heretic; upstream defaults are 100 trials with
    400-prompt probes. This run fits a single CPU job.
  • The source model's own card reports a final training loss of 2.508 on 250 steps over 279k
    segments; no independent evaluation of the full-stack ability was performed by me, before or
    after decensoring. The only thing measured here is refusals, drift, perplexity and acceptance
    rate.
  • Vision behaviour was not evaluated upstream either; the projector is the base tower converted
    to F16, unchanged.

Credits

  1. Qwen team (Alibaba) — Qwen3.5-4B architecture and vision tower, Apache-2.0.
  2. Empero-AI — empero-ai/Qwen3.8-4B-Distill, the distillation all weights derive from.
  3. iBotIA / iWebRoot — iBotIA/Qwen3.8-4B-Empero-AI-FullStack,
    the full-stack fine-tune this repo quantizes. Original model card, training logs and license
    live there.
  4. p-e-w (Philipp Emanuel Weidmann) — Heretic v2.1.0.dev0
    (ARA + Optuna TPE), AGPL-3.0-or-later.
  5. mlabonne — harmless_alpaca / harmful_behaviors probe datasets.
  6. ggml-org / llama.cpp — GGUF conversion, imatrix, quantization, runtime.
  7. hUh63 — λ sweep, MTP-preserving rebuild, domain-matched calibration corpus, quantization,
    publishing.

License: Apache-2.0, inherited from the base model lineage. Decensoring and quantization add
no new restrictions. Heretic itself is AGPL-3.0-or-later; it was used as a tool, and no Heretic
source is redistributed in the weights.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-14 card: count evaluated directions from real points, not the configured list 8c72be4 13 KB
    Loading...
  2. 2026-11-14 add README.md cb436b3 12.2 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration