base_model:
- orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
license: other
license_name: qwen-community-1.0
license_link: LICENSE
pipeline_tag: image-text-to-text
tags: - qwen3.8
- qwen4-exp
- awq
- w4a16
- e4m3
Qwen3.8 Flash Next Uncensored AWQ g32 + QSA FP8 E4M3 KV
138.13 GB (128.65 GiB) across 44 model shards: 10 AWQ, 33 PLE, and one K/V-scale shard.
AWQ derivative of orcarouter/Qwen3.8-Flash-Next-Uncensored.
Only routed-expert projections are quantized. PLE ngram weights are reused from
the fixed official FP8 checkpoint, and newly calibrated QSA FP8 E4M3 K/V scales
are included. This is a different mother checkpoint from
our original AWQ g32 release,
not a rename of its weights or a reuse of its K/V scales.
Checkpoint
- Mother revision:
8336e613ea508b13c2159bd0f68965d97a606b95 - Official FP8 PLE revision:
bcd9f01ddc9cff2316eb84281bebcd5b058bddce - Routed experts: asymmetric AWQ W4A16, group size 32, zero point, GEMM layout
- Quantized: per-expert
gate_proj,up_proj, anddown_projacross 48 layers and 512 experts per layer - Other parameters remain at source precision, except the explicit official FP8 PLE splice
- Indexed tensors: 222,771, including 24 FP32 scalar K/V scales
- Merged index SHA-256:
f066a0154a9101b359c3a4d4fa6a83fb12b7126a9cf0f19cb772611d08cc07ee - K/V-scale shard SHA-256:
a709ed27825e1b5e97e8b2127671e0d0b7715f014c5db71cc414a3b96af76be1
Identical index hashes can describe the same tensor names and shard layout in
different checkpoints; they do not prove identical weights. Use SHA256SUMS
for the complete file identity. Public provenance and results are inMODEL_PROVENANCE.json and VALIDATION.json.
Why AWQ g32
W4A16 provides a weight-only INT4 path with 16-bit activation/compute precision.
The native g32 layout also matches width-640 routed experts under TP4:640 / 4 = 160, which is divisible by 32 but not 128. Weight-fit comparisons
published for the original mother model are not measurements of this derivative
and are not reproduced here.
Calibration
AWQ weight calibration used 684 records. The existing calibration corpus and
frozen recipe were reused; this mother's weights were quantized separately.
K/V-scale calibration was run afresh on this AWQ checkpoint: 14 input shards,
4,904 prompts and 4,888,771 processed prompt tokens. Corpus and tokenizer assets
were checked before reuse. The 24 finite, positive FP32 scalar scales cover K/V
at all 12 QSA layers, ranging from 0.0170375295 to 0.0819614977.
The E4M3 runtime loaded all 24 entries. These are KV-storage scales, not AWQ
weight scales or PLE scales; scale collection and weight quantization are separate.
Loading requirements
The loader must support:
- Qwen4Exp per-expert asymmetric AWQ W4A16 g32 in GEMM layout;
- official FP8 PLE weights and indexed reuse of complete PLE shard files while ignoring unindexed tensors;
- the 24 bundled QSA K/V scale entries through
model.safetensors.index.json; - complete Qwen4Exp multimodal handling, including MRoPE and video-token-pruning initialization, for images and videos.
Validation used four 32 GiB NVIDIA V100 GPUs, TP4, MTP disabled and FP16 compute.
On V100, E4M3 refers to KV storage, not native FP8 attention compute.
Request fp8_e4m3 explicitly in a compatible runtime; merely downloading the
scales does not enable FP8 KV. Generic AWQ support alone is insufficient for this
architecture. No stock-runtime compatibility or default launch configuration is implied.
Validation
Both columns use this same AWQ checkpoint; only the KV dtype changes within each
paired test. These are small regression subsets, not full benchmark scores or
a BF16-versus-AWQ evaluation.
| Check | FP16 KV | Calibrated E4M3 KV |
|---|---|---|
| Basic generation | 4/4 | 4/4 |
| GSM8K five-shot subset, reviewed | 29/32 | 29/32* |
| HumanEval/MBPP functional subset | 9/10 | 9/10 |
| Official IFEval strict/loose | 2/5 prompts, 8/12 instructions | 2/5 prompts, 8/12 instructions |
| Tool-name selection, 128-token cap | 9/12 | 9/12 |
| Tool-name selection, 512-token cap | 8/12 | 9/12 |
| 128K needle retrieval, three positions repeated twice | 6/6 | 6/6 |
| Image requests | 8/10 | 8/10 |
| Video requests | 8/8 | 8/8 |
| Image/video total | 16/18 | 16/18 |
*The E4M3 reviewed score includes one approved truncation adjudication. On
GSM8K case 0007 the original 256-token response was cut off during a correct
calculation; the last-integer scorer extracted 100 rather than the expected 160.
A supplemental run with a 2,048-token cap returned 160 and stopped after 209 tokens.
The original capture remains 28/32. This was not a uniform higher-budget rerun
of all 32 cases and is not interchangeable with a fixed-budget 29/32 result.
The public text suite used 256 output tokens for GSM8K and 768 for code/IFEval,
with thinking disabled. All returned selected/top logprobs in both arms were finite;
this is not an audit of every internal activation. One MBPP response was truncated.
Tool tests score tool-name selection, not actual tool execution or failure recovery.
The multimodal suite contained five image and four video cases, each repeated
twice; both arms omitted shape names in the spatial image question. All 18
requests per arm returned successfully and stopped normally. Follow-up prompts
use each arm's own preceding answer.
Notes and limitations
- Same-mother BF16 reference validation was not performed, by maintainer decision. No measured BF16-to-AWQ capability loss is claimed.
- This is a published quantization with the bounded validation above, not certification for every production, agent, long-output or multimodal workload.
- The upstream “Uncensored” name identifies the mother model; it is not a guarantee about refusal behavior, safety or suitability for a particular application.
- Preserve the bundled scales for this exact weight artifact; do not substitute another model's scales.
- This derivative uses the included Qwen Community License 1.0.
Integrity
Run sha256sum -c SHA256SUMS after downloading. The checksum list covers model
shards, tokenizer/processor assets, configuration, license, this card and public metadata.