license: other
license_name: qwen-community-1.0
license_link: LICENSE
library_name: gguf
base_model:
- Qwen/Qwen3.8-Flash-Next
- huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
- unsloth/Qwen3.8-Flash-Next-GGUF
- bartowski/Qwen3.8-Flash-Next-GGUF
- SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF
base_model_relation: merge
pipeline_tag: text-generation
tags: - uncensored
- heretic
- gguf
- llama.cpp
- moe
- qwen3.8
- abliterated
- tensor-transplant
- long-context
- single-gpu
QFN-C: Qwen3.8-Flash-Next, assembled for one 32 GB GPU (GGUF)
Built on abliterated (refusal-removed) tensors. The non-expert tensors of this model, including the 96 tensors
edited by abliteration, come fromhuihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF.
It is not the original Qwen model, and its refusal behaviour was not evaluated.
A 100.14 GiB GGUF build of Qwen/Qwen3.8-Flash-Next (176.94B
parameters) that decoded 41.6 tokens/s with 518,000 tokens in a 524,288-token window on one RTX 5090, at a mean KL
divergence of 0.0572 (93.46 % same top token) from a near-lossless reference build. Nothing was trained,
re-quantized or pruned: every tensor is copied byte for byte from one of four public GGUF distributions.
Conditions:
- Speed: 41.57 tok/s at 517,811 tokens in context, every routed expert page-locked in host memory, q4_0 KV cache,
batch 2,048, 15,197 MiB expert cache on the GPU, YaRN x2, one request, patched llama.cpp (see the GitHub
repository), host with 96 GB RAM and a PCIe generation-4 slot. One run. At the 262,144-token window: 46.94 tok/s at
2K and 46.72 at 255K. - Quality: against QFN-R (same non-expert tensors, every routed expert at Q6_K or Q8_0), not against full
precision; twelve 4,096-token chunks, native positions, f16 cache. With YaRN x2 and q4_0 cache: KL 0.0759, same top
92.31 %. No task benchmark (mathematics, coding, refusal) was completed. - 128 GB host (since 2026-11-15): with every routed expert page-locked, 40.58-41.92 tok/s after a 2K prompt and
40.58-43.37 after 16K in a 3-round A/B (+14 to +20 % and +7 to +17 % over the 96 GB placement); 45-46 tok/s after
2K as served. Details indocs/PERFORMANCE.mdof the GitHub repository. - Needs: one 32 GB GPU and 96 GB RAM (69.5 GiB of model data held in RAM, 57.2 GiB page-locked) or, for the
faster all-pinned placement, 128 GB (about 68 GiB page-locked); Windows 11 as measured. The measured speeds depend on the patched engine (sparse quantized-KV attention, experts inCUDA_Hostwith an expert cache); whether unpatched llama.cpp loads these files was not tested.
Files
| file | bytes | SHA-256 |
|---|---|---|
QFN-C-00001-of-00002.gguf |
78,722,102,464 | 0d6dc9ee8a876ceb40ed01bc6f0d1e7f6ca511296c67fc9a59d8f586a31c0bd1 |
QFN-C-00002-of-00002.gguf |
28,800,138,432 | 316b46f3a2dbd68c900f43136ab9449f9dcc3725dfd8c794847c204bc161e113 |
QFN-C.manifest.json |
417,010 | tensor plan: donor file, donor offset and output offset of all 1,223 tensors of shard 1 |
QFN-C.journal.jsonl |
250,616 | SHA-256 of the 1,707 payload chunks of shard 1 |
Shard 2 is the hashed n-gram embedding table, byte-identical toSC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF IQ2_XS/...-IQ2_XS-00002-of-00002.gguf at revision90a3350cd800ec1fe0d1660a38414ad9e985782f.
How it is assembled
| tensors | donor (pinned revision) |
|---|---|
| routed experts of 18 blocks chosen by sensitivity | bartowski/Qwen3.8-Flash-Next-GGUF IQ4_XS (db3a185731e162be9ffa4d52d20d7509939e60e8) |
| routed experts of the other 30 blocks | unsloth/Qwen3.8-Flash-Next-GGUF UD-Q4_K_XL (766911a6b7369840a91dbcd95f9f997acaab6cd6) |
| all non-expert tensors | huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF Q8_0 (411f1f7349c9711e6f75aa8d20e1612eb3a4d55d) |
| n-gram embedding table (shard 2) | SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF IQ2_XS shard 2 (90a3350cd800ec1fe0d1660a38414ad9e985782f) |
The recipe, the assembler (which rebuilds these files from the donors and checks them against the two hashes above),
the engine patches, the serving profile, all measurements and the reproduction procedure are in the GitHub
repository (Tvene88/qfn-c-qwen38-flash-next).
Serving (patched engine required for these speeds)
Build llama.cpp at 79bce0c with the four patches in engine/ of the GitHub repository. Patch 0004 (sparse
attention over a quantized KV cache) and patch 0003 (experts in page-locked CUDA_Host memory with a GPU expert cache)
carry the speed: at 518K tokens of context the stock attention path decoded 20.7 tok/s, the patched one 41.6.
set GGML_CUDA_FA_SPARSE_QUANT=1
llama-server -m QFN-C-00001-of-00002.gguf -ngl 999 -ot "\.ffn_.*_exps=CUDA_Host" --no-host -nr -lm none ^
--moe-cache-mib 15197 -fa on -c 524288 -ctk q4_0 -ctv q4_0 -ub 2048 -b 2048 ^
--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 262144 --jinja ...
That is the 128 GB placement; with 96 GB use -ot "blk\.[0-7]\.ffn_.*_exps=CPU,\.ffn_.*_exps=CUDA_Host". The full
argument list and a launcher (serving/start-qfn.ps1 -Placement AllPinned|Pageable) are in the GitHub repository.
Limitations
- One host measured (RTX 5090, PCIe generation 4, Windows 11; 96 GB RAM, then 128 GB from 2026-11-15); each
configuration run once. - The served 524,288-token profile with eight blocks pageable was never measured at full depth.
- The evaluation text of the distribution test is partly private; the method can be repeated, the digits not exactly.
- No independent full reproduction has been run.
- Abliterated base.
Licence
The weights are derived from Qwen/Qwen3.8-Flash-Next and are distributed under the Qwen Community License 1.0
(LICENSE), including its notice condition and the separate licence it requires for Model-as-a-Service and
AI Work Assistant businesses. Shard 2 is SC117's file (apache-2.0 on its repository, derived from the same Qwen base).
Credits
Qwen (base model); huihui-ai, unsloth, bartowski, SC117 and ISTA-DASLab (donor distributions); llama.cpp (engine).
Recipe, assembler, engine patches and measurements by the owner of this repository working with Claude (Anthropic).