base_model: DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
base_model_relation: quantized
quantized_by: Raufaser
license: apache-2.0
pipeline_tag: text-generation
tags:
- hipfire
- quantized
- mq4
- lloyd
- amd
- qwen3.8
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU - hipfire MQ4-Lloyd
Three hipfire quantizations of DavidAU Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU in the MQ4G256V2Lloyd format (quant type 52). The FWHT-rotated MQ4V2 container keeps its 136-byte groups; a per-tensor 16-level Lloyd-Max codebook replaces the uniform 4-bit grid at the same bit rate. The files carry AWQ activation scales derived from a model-specific importance matrix collected on this checkpoint.
These load in hipfire. llama.cpp cannot read them.
Lineage
Qwen/Qwen3.8-27B
└── DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
└── this repo (MQ4-Lloyd format conversion + AWQ)
The weights come from the DavidAU fine-tune, not from the Qwen base. This repo changes only the numeric format.
Files
| File | Tier | Bytes | SHA-256 |
|---|---|---|---|
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU.mq4l |
base | 15,671,109,632 | 17ff0c086cb26a5f71ed48449bd78ff97e55ebee3ac579c73e59d5e02141aea0 |
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU.mq4l-pro |
pro | 16,472,669,184 | a93c4fcf5af500c2c2fca4c17e722bed7ced521467b7bd16d2a90fbd488a4c85 |
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU.mq4l-xt |
xt | 14,995,679,296 | 43a0e64a20f4e8b2a7e9cb316b91b0aa8fdea599f99357c1603916a72e2aa38c |
All three use the same codec. The suffix is a product tier: xt keeps lm_head at the base codec, base lifts embed and lm_head, pro also lifts ssm_out (linear_attn.out_proj).
Reproduction
From the BF16 safetensors with an imatrix collected on this model:
hf download Raufaser/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-hipfire-MQ4L davidau-imatrix.gguf
target/release/hipfire-quantize --input <safetensors-dir> \
--format mq4v2-lloyd --tier base \
--imatrix davidau-imatrix.gguf \
--awq --awq-fix-la-head-order \
--output Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU.mq4l.hfq
The other tiers use the same command with the corresponding --tier and output name. --awq-fix-la-head-order maps the llama.cpp imatrix's tiled linear-attention V-head entries back to HF order. The uploaded files drop the .hfq suffix.
The imatrix
No imatrix was published for this checkpoint. It was collected on themis (RTX 4090) with llama.cpp llama-imatrix over the frozen WT2 slice, from the Q5_K_S GGUF already cached for the model:
llama-imatrix -m <model>.Q5_K_S.gguf -f wikitext2-1024s-2048ctx.txt \
-c 2048 -b 2048 --chunks 128 -o davidau-imatrix.gguf --no-ppl
That is 262k tokens in 128 chunks. The file is 27 MB, SHA-256 27c83e34a4ba25ea8a11cb3ca15c1492e3d90c2b0782934dffa91a7a9dbf9304. A larger 1.27M-token mixed-corpus imatrix was tried and produced worse KLD on this WT2 evaluation, so this one was kept.
Quality
| Tier | Uniform MQ4V2, AWQ | MQ4-Lloyd, AWQ | Lloyd reduction | .kldseq SHA-256 |
|---|---|---|---|---|
| base | 0.044981 | 0.040335 | 10.3% | 4b6f55437c9dafc78e21e3c3864f384873634fad74a7cb2ab6b013ddbd241083 |
| pro | 0.042598 | 0.038053 | 10.7% | f44e992953994ceb0dc7b10da3c89d3019526d87634e1cb33363382ccf0009b8 |
| xt | 0.051776 | 0.046369 | 10.4% | 30c2cbb7a4a743cd26fb2f392712e57626884a289ad77b729156d65ba1559a9f |
WT2 slice, 24 chunks of 2048 tokens, 24,552 scored tokens. Scoring used eval_hipfire --scoring-mode prefill --max-chunks 24 --kv-mode q8 --kv-v q8 on gfx1100 (RX 7900 XTX, HIP 7.16 / TheRock 10.1), with HIPFIRE_GRAPH=0, HIPFIRE_NORMALIZE_PROMPT=0 and HIPFIRE_IU4_PREFILL=0 so both formats take the same weight route.
The teacher is DavidAU.mq5v2-pro.hfq (SHA-256 4231f0153aecda0dbc86044c7768158a3bf22ef6710f2f6f9a668c990d9734ea), quantized from the same safetensors. The native top-256 reference is davidau-wt2-native.kldref.bin (SHA-256 a9cfe8f80e9dc155b9260ac14d940e86c493df4ecdd167bc802075a93b67db33).
These numbers are comparable to each other, not to the published BF16-teacher pins in hipfire-models/qwen3.8-27b, which use a different teacher.
Running
A hipfire build with MQ4-Lloyd support (qt 52) is required.
hipfire serve Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU.mq4l --kv q8 --spec off
No MTP sidecar is included.
License and provenance
The source checkpoint declares Apache-2.0. These files are format conversions of that checkpoint. No rights to the weights are claimed.
MagnumQuant, the FWHT-rotated MQ family and the MQ4-Lloyd codebook are original hipfire work, catalogued in PRIOR-ART.md. hipfire is authored primarily by Kaden Schutt. MQ4-Lloyd was added in PR #768, and the gfx1100 Lloyd prefill and decode route in PR #1029.