license: apache-2.0
library_name: mlx
pipeline_tag: text-generation
base_model:
- DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored
base_model_relation: quantized
tags: - mlx
- qwen3_5
- mtp
- heretic
- uncensored
- 4-bit
Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO — MTP Q4 MLX
An MLX conversion of the MTP-Q4_K_M GGUF from
DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF,
with the native Multi-Token Prediction (MTP) head kept.
- Size: ~25 GB (mixed 4-bit / 6-bit affine, group size 32 — the GGUF's own Q4_K_M codes re-expressed, not re-quantized)
- Layers: 96 + 1 MTP layer
- Model behavior, prompting modes and the uncensored nature are as described on DavidAU's card.
Usage
Works in stock mlx-lm and LM Studio (the MTP weights are simply ignored there):
mlx_lm.generate --model kingchan818/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-MTP-Q4-mlx --prompt "hello"
MTP speculative decoding (~1.4–1.5× faster)
Native MTP decoding is not in released mlx-lm yet. It needs the fork from
ml-explore/mlx-lm#990:
uv init --bare && uv add "mlx-lm @ git+https://github.com/AirRunner/mlx-lm@feat/mtp-native"
uv run mlx_lm.generate --model kingchan818/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-MTP-Q4-mlx --mtp --prompt "hello"
uv run mlx_lm.server --model kingchan818/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-MTP-Q4-mlx --mtp
Validation (Apple M5 Max, greedy, 300 tokens)
| Prompt | MTP top-1 (t+2) | Drafts accepted | Output identical to non-MTP | Speed |
|---|---|---|---|---|
| Sky is blue | 77.7% | 42.0% | yes | 18.8 → 26.2 tok/s (1.39×) |
| Fibonacci code | 94.3% | 48.7% | yes | 19.0 → 29.3 tok/s (1.54×) |
| Planets | 86.0% | 45.3% | yes | 19.0 → 27.9 tok/s (1.47×) |
Control: with the MTP fc weight zeroed, top-1 accuracy drops to 0.0%, so the head is genuinely trained and correctly wired.
Sanity: "The capital of France is" → " Paris" (p=0.74); chat answers coherent with and without thinking.
How it was converted
Converted with gguf2mlx (--quant-source preserve) plus two fixes:
- MTP head kept. llama.cpp stores the MTP layer as an extra block (
blk.96,nextn.*). It is mapped back to HFmtp.*names (eh_proj→fc,enorm→pre_fc_norm_embedding,hnorm→pre_fc_norm_hidden,shared_head_norm→norm) andnum_hidden_layers=96,mtp_num_hidden_layers=1. - Linear-attention V-head order restored. With
linear_num_key_heads (16) != linear_num_value_heads (48),
llama.cpp reorders the Gated DeltaNet V heads from grouped to tiled order. gguf2mlx does not undo this, which produces
garbage output; the permutation is inverted here forin_proj_qkv(V rows),in_proj_z,in_proj_a/b,A_log,dt_bias,conv1d(V channels) andout_proj(input columns). The inverse was checked to be exact, including on
packed quantized weights.
Text-only (the vision tower / mmproj is not included).