base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
library_name: mlx-serve
license: other
license_name: qwen-community-1.0
license_link: LICENSE
pipeline_tag: image-text-to-text
language:
- en
- zh
tags: - mlx
- mlx-serve
- qwen4_exp
- moe
- sparse-attention
- ngram-embedding
- imatrix
- abliterated
- uncensored
Qwen3.8-Flash-Next-Uncensored for mlx-serve, iQ-MLX 4.7 bpw
This is orcarouter/Qwen3.8-Flash-Next-Uncensored
(abliterated bf16) packed for mlx-serve
with the widths and conversion scripts of
ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw.
The routed experts are 4-bit, quantized with an importance matrix collected on
these bf16 weights; everything else is 8-bit. The MTP head and the vision
tower are included.
[!WARNING]
Safety alignment removed. This model complies with requests the original refuses. You are responsible for how you use it and for what it generates. See orcarouter's disclaimer.
mlx-serve --model na0fu3y/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-iQ-MLX-4.7bpw --serve --mtp --ple-gpu
Download size is 100 GB. It takes about 75 GB resident without --ple-gpu,
and about 30 GB more with it, because the n-gram table then stays in memory.
Measured
Measured on an M3 Ultra with mlx-serve ccedc6a4, against ddalcu's pack.
The two packs were served one at a time in A-B-B-A order.
| this pack | ddalcu's pack | |
|---|---|---|
| decode tok/s, 3 short prompts (run 1 / run 2) | 124.3 / 124.8 | 126.2 / 122.9 |
| whole-file edit tok/s (run 1 / run 2) | 199.5 / 198.1 | 199.9 / 191.8 |
| needle recall of 40 at 65K / 131K / 197K | 40 / 40 / 39 | 40 / 40 / 39 |
| refusals of 13 borderline prompts | 1 | 11 |
These are short probes. Agreement with the bf16 logits was not measured.
Widths
| tensors | width |
|---|---|
| routed experts | 4-bit, group 64, imatrix-weighted |
| attention, GDN, hyper-connections, indexer, shared experts, lm_head | 8-bit, group 64 |
| embed_tokens | 4-bit, group 64 |
n-gram table (ngram_table.bin) |
4-bit, group 32 |
| routers, gates, norms, convs, SSM state, vision tower | bf16 |
ngram_table.bin is byte-identical to ddalcu's: the abliteration editsple.value_proj and embed_tokens but not the n-gram table itself.
Conversion
Uses mlx-serve's scripts at commit ccedc6a4, from orcarouter revision d9dbbb7f:
python tests/qwen38_flash_next_imatrix_collect.py --src <bf16> --out imatrix.safetensors --max-tokens 480000
python tests/convert_qwen38_flash_next.py --src <bf16> --dst <pack> --imatrix imatrix.safetensors
python tests/convert_qwen38_flash_next.py --src <bf16> --dst <pack> --add-vision
ddalcu's pack was calibrated on 1.35M tokens and this one on 484K. Nothing
measured above separates the two. The imatrix is included ascalibration/imatrix.safetensors.
License
Qwen Community License 1.0, inherited from
Qwen/Qwen3.8-Flash-Next (see LICENSE).
orcarouter's card says Apache 2.0, but the base model's license is the Qwen
Community License 1.0.