base_model:
- orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
language: - en
- zh
license: other
license_name: qwen-community-1.0
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
tags: - gguf
- llama.cpp
- qwen
- qwen3.8
- qwen4exp
- mixture-of-experts
- uncensored
- abliterated
- text-generation
Qwen3.8-Flash-Next-Uncensored — IMX-4.57bpw IQ4_NL HQ GGUF
This repository contains a custom, quality-biased GGUF quantization oforcarouter/Qwen3.8-Flash-Next-Uncensored,
which is an abliterated BF16 derivative ofQwen/Qwen3.8-Flash-Next.
This is a text-path release. It does not include the vision tower, an mmproj, or the MTP/draft head.
Release summary
| Property | Value |
|---|---|
| Upstream weights | orcarouter/Qwen3.8-Flash-Next-Uncensored BF16 Safetensors |
| Original base | Qwen/Qwen3.8-Flash-Next |
| Format | GGUF, 36 shards |
| Architecture | qwen4exp, 48 main blocks |
| Measured tensor BPW | 4.566348 |
| Total size | 101,009,466,304 bytes across the split files (94.072396 GiB) |
| Tensor count | 1,224 |
| Context metadata | 262,144 tokens |
| Included | Text model and PLE n-gram table |
| Not included | Vision tower, mmproj, MTP/draft head |
The quantization was produced directly from the upstream BF16 Safetensors, not by requantizing another GGUF. A custom importance matrix (imatrix) prepared for this upstream checkpoint was used during quantization; the matrix itself is not included in this repository.
Relationship to AtomicChat AD-4.27
The layout was inspired by and derived from AtomicChat's publishedAD-4.27bpw-Q4_K_M
recipe. Credit for the original layer-banded recipe, the IQ4_NL treatment of down-expert tensors, and the shard layout belongs to AtomicChat.
This release is not a reproduction or repack of AtomicChat's model. The important differences are:
| Area | AtomicChat AD-4.27 reference | This release |
|---|---|---|
| Source weights | Original Qwen/Qwen3.8-Flash-Next weights |
OrcaRouter's abliterated BF16 weights |
| Measured BPW | About 4.27 | 4.566348 |
| Highest-precision gate/up expert band | Blocks 0–3 and 40–47 at IQ3_S |
Blocks 0, 40–42 and 44–47 at IQ4_NL |
| Medium-precision gate/up band | Blocks 0–3 and 40–47 at IQ3_S |
Blocks 1–3, 36–39 and 43 at IQ3_S |
| Remaining gate/up experts | IQ2_S |
Blocks 4–35 at IQ2_S |
| Down experts | IQ4_NL |
IQ4_NL for all 48 blocks |
| Other tensors | Primarily Q8_0 |
Selected sensitive groups retained in BF16/F32 |
| Split | 33 files in the published AD-4.27 build | 36 files |
The protected BF16/F32 groups in this release include the QSA indexer projections, Gated DeltaNet/linear-attention weights, token embeddings, output head, Hyper-Connection weights, attention gates, shared experts, and router gates. Norms and other tensors that require or naturally retain F32 remain F32.
The PLE n-gram table remains Q5_1 and is isolated in shard 00002-of-00036, allowing it to stay mmap-backed instead of occupying accelerator memory with the ordinary model weights.
Because of these changes, do not describe this release as AD-4.27. IMX-4.57bpw-IQ4_NL-HQ is the identifying quant name.
Tensor layout
| Tensor group | Quantization |
|---|---|
blk.0, blk.40–42, blk.44–47 gate/up experts |
IQ4_NL |
blk.1–3, blk.36–39, blk.43 gate/up experts |
IQ3_S |
blk.4–35 gate/up experts |
IQ2_S |
| All down experts | IQ4_NL |
| PLE n-gram embedding | Q5_1 |
| Selected sensitive groups | BF16/F32 |
| Remaining eligible tensors | Q8_0 |
Actual tensor-type counts:
| Type | Tensors |
|---|---|
| BF16 | 640 |
| F32 | 388 |
| F16 | 1 |
| IQ4_NL | 64 |
| IQ3_S | 16 |
| IQ2_S | 64 |
| Q5_1 | 1 |
| Q8_0 | 50 |
The single F16 tensor is blk.1.ple_conv1d.weight; its four-column layout is not compatible with the requested fallback type.
Running with llama.cpp
Use a recent llama.cpp build with qwen4exp support. This release was converted, quantized, and tested with commitdaef7b6874397a5a7c3d7e38b55e2ee0adf7da38.
llama-cli \
-m Qwen3.8-Flash-Next-Uncensored-IMX-4.57bpw-IQ4_NL-HQ-M64-00001-of-00036.gguf \
--jinja \
-ngl 99 \
-c 8192 \
-fit off
Keep mmap enabled. The n-gram shard is intentionally designed to remain pageable. Always pass --jinja so the embedded chat template is applied.
Recommended sampling defaults inherited from the upstream model:
| Parameter | Value |
|---|---|
| Temperature | 1.0 |
| Top-p | 0.95 |
| Top-k | 20 |
Verification
The finished GGUF was checked programmatically:
- the n-gram table was isolated in shard 2;
- English, Chinese, basic reasoning, coding, and structured tool-call smoke tests completed successfully;
- no KLD, perplexity, or benchmark comparison against BF16 has been performed for this exact quantization.
The functional smoke tests are not a claim of benchmark parity with the BF16 model or with AtomicChat's release.
Safety and limitations
The upstream OrcaRouter model is an abliterated/uncensored checkpoint whose safety alignment has been substantially reduced. It may comply with harmful, illegal, offensive, or otherwise unsafe requests. Add appropriate moderation and access controls before any user-facing deployment. Users are responsible for complying with applicable laws and license terms.
Quantization can change model behavior and output quality. This release contains neither the vision components nor the MTP speculative-decoding head of the upstream checkpoint.
Attribution
- Upstream weights:
orcarouter/Qwen3.8-Flash-Next-Uncensored - Original base model:
Qwen/Qwen3.8-Flash-Next - Quantization recipe reference:
AtomicChat/Qwen3.8-Flash-Next-GGUF, especially its published AD-4.27 recipe - GGUF runtime and tooling:
ggml-org/llama.cpp
This repository is independently produced and is not affiliated with or endorsed by Qwen, OrcaRouter, AtomicChat, or the llama.cpp project.
License
The original Qwen repository identifies the model license as theQwen Community License 1.0.
Review that license, the OrcaRouter upstream terms, and all applicable obligations before use or redistribution.