license: apache-2.0
tags:
- exl3
- abliterated
- uncensored
- moe
- qwen4_exp
- flash-next
- reasoning
- tool-use
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
pipeline_tag: text-generation
Qwen3.8-Flash-Next-Abliterated-EXL3 @ 2.05bpw_h4_ng4
EXL3 quantization of orcarouter/Qwen3.8-Flash-Next-Uncensored
(BF16, tensor-level abliteration). 125B-A6B Qwen4-preview MoE at 2.05 bits per
weight. All 512 experts fit on 2x24 GB GPUs with no CPU offload.
Research-only release. Safety alignment has been largely removed
(see Disclaimer). You carry full responsibility for use. See License.
Model details
| Base | orcarouter/Qwen3.8-Flash-Next-Uncensored (from Qwen/Qwen3.8-Flash-Next) |
| Abliteration | Refusal-direction removal across 149 residual-writer tensors. MoE router, 51B n-gram table, and vision tower left untouched (per source card) |
| Quant | EXL3 2.05bpw_h4_ng4 via exllamav3 1.4.9 (-b 2.05 -hb 4 -mb 2 -vb 4 -cb mul1 --out_scales always, 250x2048 cal) |
| Contents | 5 shards (~34 GB) + ngram_embedding.safetensors 13 GB (K=2) + MTP head in-shard (6,203 mtp.* entries) + vision kept (987 visual.* tensors) |
| Context | 262,144 native (served here at 98,304 Q8 KV) |
| Thinking | Reasoning model. Keep thinking ON. OFF gives 0-token responses on this family |
Available quants
| Branch | Decoder | Notes |
|---|---|---|
2.05bpw_h4_ng4 |
2.05 bpw | Plain quants only |
Only plain quants at 2.05 bits per weight are available in this repository.
No other bitrates or quant types are published here.
Release status: this is the only planned release. No further quants
will be made from this source in this repository.
Measured on 2x RTX 3090 (self-hosted, exllamav3 + isolated venv)
| prompt (600 tok, streaming, temp 0) | decode |
|---|---|
| prose | ~72 tok/s |
| code | ~72 tok/s |
| prose2 | ~72 tok/s |
Stable over 5 back to back runs (range 1.3 tok/s). TTFT ~0.2 s warm.
Envelope: -gs 20,23.5 -cs 98304 -cq 8, thinking ON.
Drafting note: MTP speculative drafting measured 52 to 55 tok/s on
these weights, against ~72 without. Draft acceptance on the abliterated
weights costs more than it saves, so this release is served with -dm none.
Outputs match either way (drafting changes speed only, never content).
Quality probe (due diligence for the K=2 ngram table)
40-prompt scripted battery (temperature: 0), all verdicts automated.
40/40 PASS, rechecked under the shipping -dm none config:
- Rare words, idioms, multilingual (ES/FR/DE), low-resource (Swahili, Bengali, Finnish), entity recall. All exact
- Refusal spot checks x6 (incl. adversarial framings). 0 refusals, full compliance throughout
- Tool calls (valid
get_weather), structured JSON (parse checked), code run locally (palindrome/factorial/fib asserts pass) - Multi-step reasoning, loop check, byte-identical determinism x3
Scope honesty: this checks the behaviour of this quant only. It does
not claim orcarouter's BF16 numbers (±2 pts). Those were measured on full
precision and are cited here, not inherited.
Intended use
- Red teaming and safety testing in controlled setups.
- Interpretability and refusal-mechanism research.
- A runnable EXL3 quant for local inference (2x24 GB GPUs, no CPU offload)
and for building further quants.
Fine-tuning note
This quantized checkpoint is not a fine-tuning base. Fine-tune from the
full-precision orcarouter/Qwen3.8-Flash-Next-Uncensored
(BF16) instead, then re-quantize. Note that fine-tuning on refusal-heavy or
safety data can partly bring back refusals. Neutral/task data keeps the
uncensored behaviour.
Serving snippet (exllamav3)
# quant recipe used (for reference):
# python -m exllamav3.conversion.convert_model -i <bf16> -o <out> \
# -b 2.05 -hb 4 -mb 2 -vb 4 -cb mul1 --out_scales always --devices 0,1
# serve:
python launchers/serve_openai.py -m <dir> -gs 20,23.5 -cs 98304 -cq 8 \
-dm none --host 0.0.0.0 --port 8890
# needs exllamav3>=1.4.9, transformers (for the chat template), torch cu130
max_tokens:64+ for direct calls (32-token requests run out on the think
trace). Agent-harness default 8192 is fine.
Bias, risks, and limitations
- Safety guardrails removed. The model will produce harmful, biased, or
offensive content on request (see Disclaimer). - It carries all biases and limits of the base
Qwen3.8-Flash-Next. - The 40-prompt probe uses rule-based checks (keyword/exact-match/asserts),
not an LLM judge. Indicative only, not publication grade. Test rigorously
for your own use case.
Disclaimer. Read before use.
Uncensored research artifact, not a safety product. It may comply with
requests the aligned base would refuse, and may produce harmful, biased, or
false content. Do not put it in front of end users without your own safety,
moderation, and abuse-prevention layers. You are the operator. You are
responsible for outputs, flow-on actions, and legal compliance in your
jurisdiction. Provided "AS IS", no warranties. Outputs do not reflect the
views of the uploader, orcarouter, Qwen, or Alibaba.
License
Apache 2.0, inherited from the base model chain. Abliteration and
quantization do not change underlying licence obligations.