license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
base_model: huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF
base_model_relation: quantized
library_name: gguf
pipeline_tag: text-generation
tags:
- gguf
- ds4
- dwarfstar
- qwen4exp
- moe
- abliterated
- uncensored
- mixed-quantization
- apple-silicon
Qwen3.8-Flash-Next-Mixed-Q3-64GB-abliterated-v2 (DwarfStar / ds4)
A DS4 (DwarfStar) GGUF of Huihui's abliterated v2 build of Qwen3.8-Flash-Next, in the Mixed-Q3-64GB layout (between the Q2 and Q4 releases in size). It runs on a 64 GB Mac. Measured on a Mac Studio M5 Ultra with 96 GB of unified memory.
⚠ Read first
- This is an abliterated ("uncensored") model. Its refusal behaviour has been deliberately weakened by the upstream author (Huihui). It may produce sensitive, controversial or inappropriate content that the original Qwen model would refuse. It is not suitable for all audiences (including minors) or for settings that require high safety.
- You are solely responsible for how you use it and for complying with applicable laws, ethical standards and the licenses below. Review generated output; research, testing and controlled environments are recommended, and direct use in public-facing commercial applications is discouraged by the upstream author.
- No safety guarantees. Provided as is, without warranty. Not affiliated with or endorsed by Qwen, Alibaba, Huihui, Unsloth, antirez or Ivan Fioravanti.
- About the upstream v2: per its README, the ablation emphasis was adjusted compared to Huihui's first version and none of the experts were ablated; it is described as suited to novel writing and programming. Refusal behaviour was not evaluated here.
- Quantized language-model weights, not a lossless copy of the original model. Quality may differ from the full-precision model.
- DS4-specific:
qwen4exparchitecture with the BF16 n-gram table stored in the file. Not expected to load in other runtimes (not tested).
File
| File | Size | Resident weights |
|---|---|---|
Qwen3.8-Flash-Next-Mixed-Q3-64GB-abliterated-v2.gguf |
153,915,375,616 bytes (143.3 GiB) | 47.96 GiB |
The file contains the 95.37 GiB BF16 n-gram table, which ds4 reads directly from disk; keep it on a fast local SSD. Its SHA-256 is in SHA256SUMS.
Memory use
From ds4's startup log, including KV cache and buffers, with --prefill-chunk 1024:
| ctx 8192 | ctx 32768 | ctx 131072 |
|---|---|---|
| ≈49 GiB | ≈50 GiB | ≈54 GiB |
Only the plain Q2 file was measured at each context length (KV + buffers: 1.47 GiB at 8192, 2.30 GiB at 32768, 5.61 GiB at 131072); these values are that plus this file's resident weights. --prefill-chunk 8192 adds about 6 GiB. The same layout without abliteration measured 49.44 GiB at --ctx 8192 on an M4 Max with 64 GB.
Quantization layout
Only the routed-expert tensors (ffn_*_exps) are mixed. There are 49 blocks; block 48 is the embedded MTP block.
| Tensor | Type | Blocks |
|---|---|---|
ffn_gate_exps, ffn_up_exps |
Q4_K | 41, 42, 44, 45, 46, 48 |
ffn_gate_exps, ffn_up_exps |
IQ2_XXS | all other blocks (43) |
ffn_down_exps |
MXFP4 | 0, 3, 4, 6–10, 14–23, 30–48 (37 blocks) |
ffn_down_exps |
Q2_K (padded 640→768) | 1, 2, 5, 11–13, 24–29 (12 blocks) |
Attention / SSM / shared-expert projections and output.weight are Q8_0; token_embd and the n-gram table are BF16; HC tensors are F16; router and norms are F32.
How it was made
No re-quantization was performed; the file is a byte-level composition of existing GGUFs.
- Source of the abliterated tensors: the
unsloth-Q8_0GGUF files ofhuihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF, revision411f1f7. - Base file: the plain (non-abliterated) Mixed-Q3-64GB, described in Krissaito470/Qwen3.8-Flash-Next-DS4-Mixed-Q3 (built from the Q2 and Q4 files of antirez/qwen3.8-flash-next-gguf with
gguf-tools/mixed/splice_mixed_expert_layers_gguf.pyfrom the ds4 repository). - In the base file, 96 tensors were overwritten in place, at the same offsets and with the same byte sizes, with the corresponding Q8_0 tensors from the Huihui v2 file:
attn_output(12 blocks),ssm_out(36 blocks) andffn_down_shexp(48 blocks). The routed-expert tensors (ffn_*_exps) were not touched, which matches the upstream statement that no experts were ablated in v2. - The same procedure on the Q4 file was checked tensor by tensor: the 96 tensors were byte-identical to the Huihui v2 tensors of the same name and all 1,160 other tensors were byte-identical to the plain Q4 file. This Mixed file was built the same way, with the same result reported for it.
general.nameandgeneral.file_typein the metadata were edited afterwards (same-length replacement;file_typeset to 1024, "unspecified") because the Q2 base's values did not describe a mix. No tensor data was changed.
Because only these 96 tensors differ, the layout, size and memory use equal those of the non-abliterated Mixed-Q3-64GB.
Usage
git clone https://github.com/antirez/ds4.git && cd ds4 && make
./ds4 -m /path/to/Qwen3.8-Flash-Next-Mixed-Q3-64GB-abliterated-v2.gguf --ctx 8192 --prefill-chunk 1024 --mtp
./ds4-server -m /path/to/Qwen3.8-Flash-Next-Mixed-Q3-64GB-abliterated-v2.gguf --ctx 8192 --prefill-chunk 1024 --mtp
Requires a DwarfStar build with the native BF16 n-gram reader (older builds expecting --ple cannot load it).
Vision
Not included. The vision encoder for Qwen3.8 Flash Next is a separate GGUF (./download_model.sh qwen38-vision in the ds4 repository, passed with --vision FILE). Image input has been confirmed to work with this file.
Measured on Mac Studio M5 Ultra, 96 GB
ds4 commit 8db1d1d (with local modifications to ds4_help.c, a test script and gguf-tools/mixed). One warm-up run discarded, median of three. No other ds4 / mlx_lm process running. The non-abliterated Mixed-Q3-64GB is shown for comparison.
| this file | Mixed-Q3-64GB (plain) | |
|---|---|---|
Decode, ds4-bench, 8K ctx (tok/s) |
68.9 | 69.2 |
Prefill, ds4-bench, 8K ctx (tok/s) |
1,916 | 1,918 |
| Decode, CLI, no MTP, prose / code (tok/s) | 73.4 / 73.4 | 73.2 / 73.4 |
| Decode, CLI, MTP, prose / code (tok/s) | 83.7 / 102.8 | 84.9 / 101.1 |
CLI runs: 256 generated tokens, greedy (--temp 0 --nothink), ctx 8192, --prefill-chunk 1024. MTP speed-up depends heavily on the content.
Agreement with a hosted reference
Average negative log-likelihood (NLL, lower is better) of continuations produced by Alibaba's hosted qwen3.8-flash (via OpenRouter), using gguf-tools/quality-testing with the bundled Qwen3.8 fixtures. Percentages are relative to the plain Q4 GGUF, which scores 0.2906 / 0.1253.
| Set | this file | Mixed-Q3-64GB (plain) |
|---|---|---|
| 100 short prompts (ctx 4096) | 0.3401 (+17.0%) | 0.3327 (+14.5%) |
| 12 long prompts (ctx 32768) | 0.1352 (+7.9%) | 0.1409 (+12.4%) |
| top-1 agreement, 100 prompts | 89.4% | 89.7% |
The reference is the hosted non-abliterated model, so the numbers for this file also reflect the distance introduced by abliteration; they are not an evaluation of refusal behaviour (not measured). The hosted model is described by the ds4 README as based on Flash Next, not as a byte-identical checkpoint. The 100 prompts have about 56 target tokens each and the long set has only 12 cases: small differences are not conclusive.
Credits
- Original model: Qwen/Qwen3.8-Flash-Next (Qwen)
- Abliteration and the Q8_0 source GGUFs (
unsloth-Q8_0, following the layout of unsloth/Qwen3.8-Flash-Next-GGUF as described in the upstream README): Huihui (huihui-ai) - DS4 GGUF format, mixing tool, n-gram packaging, engine: Salvatore Sanfilippo (antirez) / DwarfStar; non-abliterated Q2 / Q4 tensors used for the base file: Ivan Fioravanti
- Layer mixing, composition of the v2 tensors and this release: Krissaito470
License
Distributed under the Qwen Community License 1.0 (Copyright (c) 2026 Qwen), the same license declared by the upstream huihui-ai/Huihui-Qwen3.8-Flash-Next-abliterated-v2-GGUF repository. The LICENSE file is included and its copyright and permission notice must be kept in all copies.