license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
library_name: mlx-serve
pipeline_tag: image-text-to-text
tags:

• mlx
• mlx-serve
• qwen4_exp
• moe
• abliterated
• uncensored
• imatrix
• apple-silicon

Qwen3.8-Flash-Next-Uncensored-MLX-Serve-iQ-MLX-4.7bpw

mlx-serve pack of orcarouter/Qwen3.8-Flash-Next-Uncensored, the abliterated BF16 of Qwen/Qwen3.8-Flash-Next (qwen4_exp). Routed experts and embeddings are 4-bit, the other matrices 8-bit, with imatrix-weighted affine quantization (iQ-MLX); the importance matrix was collected on the abliterated weights. The tensor layout and widths are those of ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw.

Lineage

1. Qwen/Qwen3.8-Flash-Next: BF16, 179,999,981,459 parameters.
2. orcarouter/Qwen3.8-Flash-Next-Uncensored, per its card: abliteration following Arditi et al. (2024). One refusal direction r is taken from the layer-24 block input (2560-d) and projected out of the 149 residual-writing tensors, W' = W - r(r^T W) in float32: 13 self_attn.o_proj, 36 linear_attn.out_proj, 49 mlp.experts.down_proj, 49 mlp.shared_expert.down_proj, ple.value_proj, embed_tokens. Untouched: the vision tower, routers, experts.gate_up_proj, hyper-connection mixers, the QSA indexer, the n-gram table, mtp.fc_*, norms, lm_head. Reported refusal on BF16 with vLLM, thinking off: AdvBench 100.0% -> 2.0%, JailbreakBench 94.0% -> 0.0%, HarmBench 100.0% -> 1.3%; XSTest over-refusal 9.6% -> 1.2%.
3. This pack: the mlx-serve converter at 431ac5d, with the imatrix collected on step 2.

Files

Total: 107,326,631,274 bytes in 111 files. 12 files (33.6 GB) are byte-identical to ddalcu's pack: ngram_table.bin, model-vision.safetensors, model-00100.safetensors, the config, index, tokenizer and template files and LICENSE. The other 99 shards differ (abliterated weights, different imatrix). ngram_table.bin is ddalcu's file: abliteration leaves the table untouched, and it is quantized without imatrix.

Quantization

MLX affine.

• imatrix-weighted: 722 tensors. Plain min-max: 72 tensors without statistics, among them embed_tokens, lm_head, the final hyper-connection mixer and the MTP head; also, inside the weighted expert tensors, the 8 experts that calibration never routed to (7 in layer 0, 1 in layer 47).
• Safetensors: 128,799,735,699 parameters in 75,302,793,208 bytes, 4.68 bits per weight. With the n-gram table: 179,999,981,459 parameters in 107.30 GB, 4.77 bits per weight.

Architecture

From config.json:

Build

imatrix, on 4 Macs, one quarter of the corpus each (--shards / --shard is a 13-line local patch: shard i takes sequences i::4; partial sums and token counts are added):

python tests/qwen38_flash_next_imatrix_collect.py --src <abliterated BF16> \
  --out imatrix_s$i.safetensors --max-tokens 1350000 --shards 4 --shard $i --device mps

The corpus is the collector's own (--seq-len 4096, --seed 20260912), rendered with the model's chat template, held-out split excluded: 1,239,110 tokens in 995 sequences (agent traffic 338,748, code 337,764, math 224,691, prose 337,907).

Conversion:

python tests/convert_qwen38_flash_next.py --src <abliterated BF16> --dst <pack> \
  --imatrix imatrix.safetensors --bits 4 --expert-gs 64 --nonexpert-bits 8 \
  --embed-bits 4 --ngram-bits 4 --ngram-link <ddalcu pack>/ngram_table.bin --jobs 26
python tests/convert_qwen38_flash_next.py --add-vision --src <abliterated BF16> --dst <pack>

Validation

2026-10-06, mlx-serve 431ac5d, Mac Studio M3 Ultra 512 GB. Reference: BF16 next-token logits of the abliterated model at 4 positions per held-out window (collector --reference), scored through mlx-serve with tests/qwen38_flash_next_score.py. Comparison pack: Qwen3.8-Flash-Next-Uncensored-MLX-Serve-mixed-4-8bit (as on yomie4343/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-mixed-4-8bit), the same layout and widths, min-max quantization without imatrix.

KL is KL(BF16 || pack). Paired, on the 454 positions both packs scored:

• KL 0.1015 vs 0.1628, difference -0.0613, 95% CI [-0.1013, -0.0273].
• Top-1 90.5% vs 90.5%, McNemar p = 1.000 (20 positions right only in this pack, 20 only in the comparison).
• Reference probability of the pack's token 0.7001 vs 0.6948, difference CI [-0.0035, +0.0139].
• Top-1 where the reference argmax has p >= 0.3: 95.1% vs 92.5% (16 vs 6 discordant positions, p = 0.052).
• On math, all 6 discordant positions favour the comparison pack; there the reference argmax has a median probability of 0.109 (0.980 where both packs agree). At p >= 0.3 both packs score 100% on math (n = 58).

Other checks:

• Abliteration after quantization: the projection of 13 dequantized residual writers (layers 0, 4, 24, 47 and MTP) onto r, relative to 16 random directions, has a median of 0.011x and a maximum of 0.090x (the 4-bit down_proj). The comparison pack: 0.012x and 0.096x.
• Agent loop (a fixed tool loop on a mock repository, 15 runs): the longest run of identical consecutive tool calls was 1 in 14 runs and 2 in 1 run. The comparison pack: 1 in all 15.
• Decode, greedy, 256 tokens: with MTP 117.6 tok/s against 118.5 tok/s for the comparison pack (medians of 6 alternating A/B cycles, ratio 0.991); without MTP 70.8 vs 71.6 tok/s. Other models were serving on the same node during these runs.
• Refusal rate was not measured on this pack.

Running

mlx-serve --model AinsleyMarkov/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-iQ-MLX-4.7bpw --serve

• Memory: about 75 GB resident (100 shards and the vision tower) plus the KV cache. mlx-serve memory-maps ngram_table.bin and dequantizes its 16 rows per token.
• Thinking is on by default; "enable_thinking": false turns it off. Tool calls use Qwen3.8's XML format.
• generation_config.json: temperature 1.0, top_p 0.95, top_k 20.

License

Qwen Community License 1.0 (LICENSE), inherited from Qwen/Qwen3.8-Flash-Next.