license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
base_model:
- orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
pipeline_tag: text-generation
tags: - gguf
- ik_llama
- qwen4exp
- imatrix
- mtp
- speculative-decoding
- text-only
- abliterated
- uncensored
Hugging Face's download widget does not recognize _KT GGUFs. Download links are in the table below.
Qwen3.8-Flash-Next-Uncensored, ik_llama.cpp IQ4_KT_v2 GGUF
Text-only GGUF of orcarouter/Qwen3.8-Flash-Next-Uncensored, an abliterated build of Qwen3.8-Flash-Next, plus an IQ4_KT quant of that model's own MTP draft head.
The recipe and importance matrix are the ones used for the base-model IQ4_KT_v2, so the two releases differ only in the weights they were quantized from.
The source model has its safety alignment removed, and quantization does not restore it. Read the source card's disclaimer before use.
These files require ik_llama.cpp at cb860a9 or later, the merge of PR #2481. Earlier builds read the IQ4_KT tails in these files incorrectly.
Files
| File | Quant | Download |
|---|---|---|
Qwen3.8-Flash-Next-Uncensored-ik_llama-IQ4_KT_v2-00001-of-00002.gguf |
IQ4_KT_v2 | 44.41 GiB |
Qwen3.8-Flash-Next-Uncensored-ik_llama-IQ4_KT_v2-00002-of-00002.gguf |
IQ4_KT_v2 | 36.31 GiB |
| IQ4_KT_v2 total | 80.72 GiB | |
Qwen3.8-Flash-Next-Uncensored-MTP-ik_llama-IQ4_KT.gguf |
IQ4_KT | 2.02 GiB |
Download both shards into the same directory and pass the first shard to llama-server.
The PLE table and the routed expert gate/up tensors use IQ4_KT. ffn_down_exps and ffn_down_shexp use IQ3_KT. Token embeddings use Q8_0. The four-column PLE convolution uses F16.
MTP draft head
The head is optional. Load it beside the main model with -md and enable it with --spec-type mtp:n_max=1,p_min=0.0.
Runtime
Example invocation:
GGML_CUDA_NO_PINNED=1 llama-server \
-m Qwen3.8-Flash-Next-Uncensored-ik_llama-IQ4_KT_v2-00001-of-00002.gguf \
--defer-ple -ngl 49 -ot 'ffn_.*_exps=CPU' \
-c 131072 -b 2048 -ub 2048 -fa on -ctk q8_0 -ctv q8_0 -ictk q8_0 --jinja
With the head, add -md Qwen3.8-Flash-Next-Uncensored-MTP-ik_llama-IQ4_KT.gguf --spec-type mtp:n_max=1,p_min=0.0.
--defer-ple leaves the 26.8 GiB PLE table on disk. The routed experts are read every token and need about 51 GiB of RAM or VRAM. Metal, Vulkan, ARM, and multi-GPU execution are not validated.
Validation
KL divergence against the BF16 source
The reference is a BF16 GGUF converted from the same uncensored weights. The test texts, chunking, and flags are the ones used for the base-model release. Each text is ten chunks of 2048 tokens, one of held-out English and one of CPython and PostgreSQL source.
| File | Reference | English mean KLD | Same top-1 | Code mean KLD | Same top-1 |
|---|---|---|---|---|---|
| This IQ4_KT_v2 | Uncensored BF16 | 0.132232 | 86.500% | 0.126008 | 93.109% |
| Base-model IQ4_KT_v2 | Base BF16 | 0.132805 | 86.276% | 0.124558 | 93.216% |
Reference perplexities were 3.3289 for English and 1.7258 for code. This file measured 3.5398 and 1.7962. Same top-1 is the share of positions where the quantized model's most likely token matches the reference's.
Calibration
Both importance matrices were computed on the base Qwen3.8-Flash-Next, not on the abliterated weights. The main model used qwen4exp-agentic-v3.imatrix and the head used qwen4exp-mtp-v1-draft.imatrix. Both files and a description of the corpus are in the base-model repository.
Conversion notes
Source: orcarouter/Qwen3.8-Flash-Next-Uncensored at revision 8336e613ea508b13c2159bd0f68965d97a606b95.
The BF16 conversion used convert_hf_to_gguf.py from llama.cpp at f8dbcd6. The head was converted with the same script and a local patch that exports the mtp.* tensors, which the stock converter omits. Both files were quantized with llama-quantize from ik_llama.cpp at f3d6e6e.
Main model options:
--imatrix qwen4exp-agentic-v3.imatrix
--token-embedding-type q8_0
--custom-q "ffn_down_exps=iq3_kt,ffn_down_shexp=iq3_kt"
iq4_kt
Head options: --imatrix qwen4exp-mtp-v1-draft.imatrix, iq4_kt.
Tensor counts in the main model are 713 IQ4_KT, 96 IQ3_KT, 25 Q8_0, one Q6_K, 388 F32, and one F16, the same as the base-model IQ4_KT_v2.
SHA-256 of the files as uploaded:
fd901716b220edd6a8bf0ce5e76df6e1b2c2ad85913c7e58d5ceb1a26ff1e426 Qwen3.8-Flash-Next-Uncensored-ik_llama-IQ4_KT_v2-00001-of-00002.gguf
0fec818e5768ea98cd539c92bc2f2bf0027c40ff9c1a4d866f65984cb35da0ad Qwen3.8-Flash-Next-Uncensored-ik_llama-IQ4_KT_v2-00002-of-00002.gguf
c05f8eb264162fce3dbd5210c33045c089afa43db3997efe1562b191ad5348ef Qwen3.8-Flash-Next-Uncensored-MTP-ik_llama-IQ4_KT.gguf
License
Qwen Community License 1.0, the license of Qwen3.8-Flash-Next. The source repository ships the same LICENSE file, and this repository includes a copy.