license: apache-2.0
base_model: HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
base_model_relation: quantized
tags:
- mlx
- mtplx
- mtp
- multi-token-prediction
- qwen3.8
- uncensored
- 4-bit
library_name: mlx
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTPLX-4bit
MLX 4-bit build of HauhauCS's Qwen3.8-27B Uncensored Aggressive fine-tune with a working multi-token-prediction (MTP) head, forged and verified with MTPLX (contract=true).
At ~16 GB this fits 24/32 GB Macs. If you have 32 GB+ and want higher fidelity, use the 6-bit build.
Measured speed (M-series Mac, 48 GB)
| Mode | decode tok/s | vs AR |
|---|---|---|
| Autoregressive | 14.9 | 1.00x |
| MTP depth 1 | 31.0 | 2.08x |
| MTP depth 2 | 37.3 | 2.51x |
| MTP depth 3 | 32.5 | 2.18x |
MTPLX auto-tunes the depth per machine; on the build machine depth 2 was selected.
Usage
mtplx pull <this-repo>
mtplx tune --model <this-repo> --retune # calibrate MTP depth for your machine
Provenance and deliberate changes
This is a community conversion, not an official HauhauCS release (their releases carry Ed25519-signed manifests; this one does not).
- Quantization lineage: original bf16 → author's Q8_K_P GGUF → dequantized f16 → MLX 4-bit affine quantization (group size 64). Note this stacks a third quantization layer on an ablated fine-tune; it passed MTPLX Forge's output-quality verification, but for maximum fidelity prefer the 6-bit build. The MTP head is kept in bf16 (
mtp_policy: keep_bf16) and was verified against the trunk. - Conversion pipeline: llama.cpp's qwen35 GGUF applies a tiled V-head reorder to the linear-attention tensors and stores
A_logas-exp(A_log); both transforms were inverted to restore the HF/MLX layout before forging. - Chat template: thinking is disabled by default (the original template defaults to enabled). Pass
enable_thinking: trueto turn it back on. - Tokenizer: rebuilt from the family reference tokenizer to restore
<think>/</think>special-token registration, which the GGUF metadata round-trip loses.
Credits
- Base model: Qwen/Qwen3.8-27B (Apache-2.0)
- Fine-tune: HauhauCS
- MTP runtime and Forge: MTPLX