license: other
license_name: qwen-community-1.0
license_link: LICENSE
library_name: mlx
pipeline_tag: image-text-to-text
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
tags:
- mlx
- mlx-serve
- qwen4_exp
- apple-silicon
- mixed-precision
- uncensored
- mtp
Qwen3.8-Flash-Next-Uncensored — MLX-Serve mixed 4/8-bit, verified mirror
This is a verified mirror of existing weights, with a reproducible local
mlx-serve runtime patch and measurements on a Mac Studio M3 Ultra / 96GB.
It is not a newly trained model, a new abliteration, or a new quantization recipe.
All 102 weight/N-gram files (107,303,406,192 bytes) were compared by SHA-256
against ARC4NUM's existing pack
at revision 9ebf9993b1eaec96aec938bf883601b51a90393b: 102/102 matched.
The model configuration also matched. The original pack deserves the credit;
this mirror adds the local runtime work and validation notes below.
The MTP fix changes the inference engine, not the model weights. Downloading
these weights alone does not install the fix. See engine/README.md.
Lineage and contents
- Base model: Qwen/Qwen3.8-Flash-Next.
- Uncensored source: orcarouter/Qwen3.8-Flash-Next-Uncensored,
local conversion source revision8336e613ea508b13c2159bd0f68965d97a606b95. - Existing byte-identical pack: ARC4NUM.
- Engine and converter: ddalcu/mlx-serve.
- This repository: mirror, local runtime modifications, and validation by yomie4343.
| Component | Storage |
|---|---|
| Routed MoE experts | 4-bit affine, group 64 |
| Other eligible projections | 8-bit affine, group 64 |
| Token embeddings | 4-bit affine, group 64 |
| N-gram table | 4-bit, group 32, separate SSD-mapped file |
| MTP head | Included, following the same mixed policy |
| Vision | Source vision weights preserved |
This is an mlx-serve-specific pack with a separate ngram_table.bin.
Do not assume generic Transformers, mlx-lm, or mlx-vlm can load it directly.
Keep the complete pack on a fast SSD and allow roughly 108GB plus working space.
Download and run
hf download yomie4343/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-mixed-4-8bit \
--local-dir ./qwen-flash-next
Build the pinned, patched runtime using engine/README.md.
Then, from that engine checkout, a conservative text-only starting configuration is:
MLX_SERVE_CACHE_LIMIT=2147483648 MLX_SERVE_NGRAM_WARM=0 \
./zig-out/bin/mlx-serve --model /absolute/path/to/qwen-flash-next \
--serve --host 127.0.0.1 --port 11234 --ctx-size 32768 \
--kv-quant 8 --prefill-chunk 1024 --no-mtp --no-pld
The starting configuration above is deliberately not the full benchmark profile.
For the exact tested MTP-auto settings, see evaluation/README.md.
No wired-memory-limit increase or memory-admission bypass is recommended.
Only expose the loopback endpoint externally after adding authentication and
appropriate access controls.
Measured results, 2026-09-08 JST
Same patched engine, same 25 TypeScript tasks, temperature 0, thinking off,
maximum output 3072, 12 agent iterations. Both models were independently regraded.
| Model | Passed | Task execution + normal grading |
|---|---|---|
| Original model | 22/25 (88%) | 38m 13.478s |
| This Uncensored model | 21/25 (84%) | 36m 50.870s |
The Uncensored run took about 3.6% less wall time, but solved one fewer task.
This is one trial per model, not evidence of general superiority, and wall
time includes tool use and revisions rather than measuring decode speed alone.
All 50 initial test outcomes matched independent grading.
The patched runs completed 27 MTP use/reload cycles without the reproduced
quantization-shape crash. A separate captured-history replay failed on request
19 with the old engine and passed 19/19 with the fix, including 14 MTP reloads.
An actual Hermes tool round trip at about 19.7K input tokens passed in 29.6s;
its cached follow-up decoded at 53.5 tok/s. These are workload-specific observations.
Limitations and safety
- Advertised/native context is not a guarantee of usable context on 96GB.
The benchmark configured 204800, but its coding tasks did not exercise that full length. - Earlier local image-plus-200K follow-up testing failed memory admission; one
image-plus-180K test passed. Image allocations persisted during a 60-second idle check.
Those cases were not rerun as part of this MTP fix and are not capacity guarantees. - Greedy AR and MTP outputs have not been established as universally identical.
- The local patch includes experimental QSA/cache/lifecycle work; it is not an
upstream release or a promise of support on other hardware. - This source model is uncensored. Outputs can be unsafe, incorrect, or offensive.
Use sandboxing, least-privilege tools, human review, and application-level safeguards.
Do not give it unrestricted access to credentials, personal files, or external actions.
License and attribution
The weights retain Qwen Community License 1.0, not Apache-2.0.
Read its commercial-use provisions, including the separate-license condition
for qualifying Model-as-a-Service / AI Work Assistant businesses.
The runtime patches retain mlx-serve's MIT terms and applicable third-party
notices; see engine/LICENSE, engine/NOTICE,
and engine/LICENSE-APACHE-2.0.
Powered by MTPLX by Youssof Altoukhi — https://github.com/youssofal/mtplx — for
upstream-derived portions identified in the engine notices.