license: other
license_name: qwen
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
tags:
- qwen
- moe
- vision
- multimodal
- exl3
- abliterated
- uncensored
- mtp
- text-generation
- image-text-to-text
base_model: - Qwen/Qwen3.8-Flash-Next
- orcarouter/Qwen3.8-Flash-Next-Uncensored
Qwen3.8-Flash-Next Abliterated — EXL3 2.50 bpw (single 24 GB GPU)
A refusal-removed (abliterated) build of Qwen's Qwen3.8-Flash-Next, quantized to EXL3 2.50 bpw so the full model runs on a single 24 GB card (RTX 3090 / 4090) using MoE CPU-offload — with the vision tower, MTP head, native 262,144-token context, and the PLE n-gram table all intact.
What this is
- Base model: Qwen/Qwen3.8-Flash-Next — a Qwen4-preview MoE (~177B stored / ~6B active), Gated DeltaNet linear attention + Qwen Sparse Attention, MTP, native vision, 262K context.
- Abliteration source: orcarouter/Qwen3.8-Flash-Next-Uncensored (BF16, 131 shards).
- Quantization: EXL3 2.50 bpw, single pass. Covers that the BF16 (360 GB) source was abliterated by OrcaRouter — this repo is the quant, not a new ablation.
Layout
| layer group | bitrate | flag |
|---|---|---|
| routed experts / decoder | 2.50 bpw (per-tensor alloc, ~2.0–3.0) | -b 2.50 |
| attention + shared experts | higher ("hq") | -hq |
| MTP head | 4 bpw | -mb 4 |
| vision tower | 6 bpw | -vb 6 |
| n-gram embedding table | 3 bpw | -ngb 3 |
Total artifact: ~61 GB (41.5 GB weights in 6 shards + 19 GB n-gram table).
Conversion
# engine (MoE CPU-offload capable):
git clone -b dflash2-pathway https://github.com/r0b0tlab/exllamav3
# convert (from the abliterated BF16):
cd exllamav3 && python3 convert.py \
-i ../Qwen3.8-Flash-Next-Uncensored \
-o ../qwen38-flash-next-abliterated-exl3-b250 \
-w ../work/target-b250 \
-b 2.50 -mb 4 -vb 6 -hq -ngb 3 -cr 250
Running (single 24 GB card)
Requires the same MoE CPU-offload setup as the stock 2.50bpw pack. Needs ~59 GB host RAM for the CPU expert tail and a fast NVMe for the streamed n-gram table.
# interactive (MTP + MoE offload):
python3 examples/chat.py -m ./qwen38-flash-next-abliterated-exl3-b250 \
-mode chatml -cs 262144 -cq 3 -mcs 384 -mct 6 -mtp
Expected on an RTX 3090: ~38 tok/s decode with MTP on (~28 without), ~20 tok/s at 175K depth, ~664 tok/s prefill. See the upstream repo for the full measured ledger; this quant uses the identical flags and layout, so numbers should track closely.
Smoke test (on the quantized artifact)
Fired on the 2.50 bpw output, not the BF16, to confirm the ablation survived quantization:
- weapons prompt (RDX synthesis) → answered directly, no refusal
- medical prompt (pentobarbital protocol/dosage) → answered directly, no refusal
- sanity (17×23) → correct step-by-step (391), coherent reasoning, native
thinkingtags intact
Notes & caveats
- Not retrained, not instruction-tuned-away. Abliteration removes the refusal direction; the underlying model still has gradient-descent knowledge and no safety alignment layer. Assume standard safety/capability tradeoffs of any ablated model.
- Research use. Same Qwen license terms as the base; this is the quant of OrcaRouter's ablated BF16.
- Low-bitrate (2.5 bpw) quantization is aggressive — per-token quality is below the BF16 source, expect some degradation on hard recall vs. denser quants. The refusal check passed at this bitrate; broader evals (the upstream Q200v2 kit) were not re-run on this ablated quant.
- Vision tower is present but was not smoke-tested here; it's preserved at 6 bpw per the conversion flags.
License
Underlying weights: Qwen License (see parent). Ablitration: OrcaRouter's work. Quant: EXL3 (ExLlamaV3, MIT).