license: apache-2.0
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
library_name: gguf
tags:
- gguf
- llama.cpp
- qwen4exp
- flash-next
- speculative-decoding
- mtp
- quantized
- iq4_nl
- abliterated
Qwen3.8-Flash-Next-Uncensored — IQ4_NL (MTP-split)
📖 Full testing & methodology documentation — how this quant was built, every test, the testing methods, the measurement confound we caught and retracted, and the iterations we went through. Charts included.
Quality-biased IQ4_NL GGUF of orcarouter/Qwen3.8-Flash-Next-Uncensored (Apache-2.0; abliterated Flash-Next), with the 1-layer NextN/MTP head split out as a separate draft sidecar for speculative decoding. Built for long-session agentic and creative workloads — heavy reasoning, character/story, planning, games, architecture/CAD, 3D-spatial, science — and accepted only after a balanced evaluation with a replicated-loss criterion (zero replicated quality losses vs both the base model and the source Q8_0 reference).
No custom kernel required. The model runs in standard llama.cpp — qwen4exp architecture support is upstream, so any recent stock build loads the backbone. (Unlike our earlier Qwen3.6 RYS GGUFs, which required a custom loader.) The fork branch is optional: it adds MTP speculative decoding and performance paths on top.
Files
| File | Size | Purpose |
|---|---|---|
qwen38_flash_next_uncensored_raw_iq4nl.gguf |
119 GB | Backbone (48 layers, MTP head removed) |
mtp-draft-uncensored-forknames.gguf |
4 GB | NextN/MTP draft sidecar (spec-decode draft) |
mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf |
1 GB | Vision projector |
MODEL_PROVENANCE.json |
— | Pinned source revisions, transformation chain, integrity checks |
How it was prepared
- Verified source: the pinned orcarouter release (an NVFP4 derivative was evaluated and rejected during provenance verification).
- Backbone repack: 32 MTP-head tensors moved to the sidecar; 1,224 backbone tensors verified by name/shape/dtype with value sampling at start/middle/end (3,360 checks).
- Imatrix calibration: multi-domain corpus weighted toward the target workloads (926 entries).
- Mixed-precision IQ4_NL: 754 IQ4_NL, 46 Q5_K, 3 Q6_K, 32 Q8_0, 388 F32, 1 F16.
Quality
Symmetric balanced evaluation — identical prompts, samplers, and seeds across variants, 36 suite-seed runs, responses scored on a 10-point rubric judged blind; a loss only counts when it replicates across seeds:
| Variant | Size | Mean score | Thinking | Zero replicated losses |
|---|---|---|---|---|
| This IQ4_NL | 119 GB | 6.33 | 10/10 | — |
| Base model | ~340 GB | 6.00 | 10/10 | reference |
| Source Q8_0 reference | 188 GB | 5.67 | 10/10 | reference |
Mean differences of a few tenths of a point sit within the suite's margin of error — read the table as parity, not a ranking. The claim this evaluation supports is no noticeable quality loss: zero replicated losses against both references, at 63% of the Q8_0 reference's size. If you need a stronger guarantee for your use case, the documentation page publishes the full methodology so you can run your own.
Usage
The backbone runs on upstream llama.cpp master (qwen4exp architecture is upstream). MTP speculative decoding needs the fork branch flashnext-mtp:
llama-server \
-m qwen38_flash_next_uncensored_raw_iq4nl.gguf \
--model-draft mtp-draft-uncensored-forknames.gguf \
--spec-type draft-mtp,ngram-map-k \
--mmproj mmproj-Qwen3.8-Flash-Next-Uncensored-F16.gguf \
-c 262144 -fa on
Measured on a 7-GPU mixed rig (262k ctx, fp32 KV): prefill on fresh 25k-token prompts — MTP draft alone 424 tok/s, MTP + n-gram 437 tok/s (+3%); decode as 600-token continuous generations — 58.8 tok/s (+8%). A 1-deep draft measured best (deeper chains from the 1-layer head decay acceptance monotonically; depth sweep ran on a fixed ≈40-token chat prompt, context under ~700 tokens — short-context decode, expect lower tok/s as context fills).
Hardware caveat: our testbed's PCIe links run at Gen3 x4, which makes multi-GPU layer-split decode interconnect-bound. On Gen4/Gen5 systems (or fewer/faster cards) expect better speeds — treat our numbers as a conservative floor.
Acknowledgments
- Alibaba's Qwen team — the Flash-Next architecture and base model.
- orcarouter — the uncensored release this quantization is derived from, and the direct source of the verified weights.
- ggml-org/llama.cpp — the runtime, including upstream qwen4exp support.
- flash-linear-attention (GDN l2norm reference) and unsloth (draft-head GGUF layout).
License
Apache-2.0, inherited from the base model.