license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:

• nvfp4
• fp8
• modelopt
• sglang
• blackwell
• uncensored
• abliterated

Qwen3.8-Flash-Next-Uncensored-NVFP4 (beginning-ai)

A mixed-precision quantization of
orcarouter/Qwen3.8-Flash-Next-Uncensored
(revision e096800036ec20da7e2442dcd4044a004d4e99fa), made to run on one 96 GB
Blackwell GPU with SGLang. That model is an abliterated (refusal-removed) BF16 build
of Qwen/Qwen3.8-Flash-Next.

This checkpoint has the same structure, formats and file layout as our censored
quant, beginning-ai/Qwen3.8-Flash-Next-NVFP4.
The two can be swapped by changing only the model path. Its activation scales were
measured again on this model.

• 120 GiB on disk. The BF16 source is 335 GiB.
• About 78 GB of weights on the GPU. The 47.7 GiB n-gram PLE table stays in host
  RAM or on NVMe.
• On one RTX PRO 6000, single-stream decode runs at about 265 tok/s at 2K
  context and about 240 tok/s at 200K, with a 786K-token KV cache and 524K
  context.
• Nothing is removed: the vision tower and the multi-token-prediction (MTP)
  draft layer are kept.

Warning. The source model has had its refusal behavior removed. It will follow
harmful, unethical or illegal requests that the original model refuses, and it has
no built-in guardrails. Use it only where you add your own safeguards, and follow
the license and the laws that apply to you. We did not change or measure its
refusal behavior; see the source model card.

How this differs from the source and the base

From Qwen's base to orcarouter's source. We compared every tensor with the
official base. orcarouter changed 149 of the 1,658 tensors, all of them weights
that write into the residual stream:

• the output projections of all 36 linear-attention layers and 12 sparse-attention layers;
• the routed-expert and shared-expert down projections in all 48 layers;
• the token embeddings and one PLE projection;
• 3 tensors in the MTP layer.

The router, the norms, the vision tower, the output head and the 47.7 GiB n-gram
table are byte-identical to the base.

From orcarouter's source to this checkpoint:

The full per-layer list is in hf_quant_config.json (quant_algo: MIXED_PRECISION).
Byte counts and the calibration report are in MANIFEST.json.

How it was made

• Quantizer. NVIDIA ModelOpt 0.47.0 tensor quantizers (NVFP4QTensor,
  FP8QTensor), applied to orcarouter's BF16 weights.
• Shared scale for gate and up. SGLang runs each expert's gate_proj and
  up_proj as one fused GEMM with one global scale. So both halves are quantized
  against one shared weight_scale_2, as ModelOpt does for fused layers. A build
  check confirms that all 24,624 fused pairs, plus the 512 MTP experts, have equal
  scales.
• Activation scales, measured on this model. We served a first build of this
  model in SGLang with a forward hook on every routed-MoE layer. The hook recorded the
  MoE inputs and routing for 16 real coding-agent prompts (332,244 tokens) and saved
  100,123 rows per layer. From that capture:
  • gate_proj/up_proj: one scale per layer, from the largest MoE input over
    every token.
  • down_proj: one scale per expert, from that expert's own activations
    (silu(gate) * up), computed with the BF16 weights on the tokens routed to it.
    24,184 of the 24,576 experts had at least 32 routed tokens; the median is about
    1,350. The rest take the larger of their own maximum and the layer's 95th
    percentile.
  • MTP experts: one scale per layer, from the same capture.
  • No scales were taken from another release.
• No quantization of the output head. It stays BF16, as in NVIDIA's and
  RadixArk's releases of the base. A 4-bit head changed the top token at about 20%
  of positions in our tests.

Results

Test machine

Correctness checks

Coding agent: DeepSWE smoke test

DeepSWE v1.1 with the mini-swe-agent harness,
reasoning effort medium, 524K context. We ran 4 tasks that our censored quant
solves, one attempt each, to check that the model works as an agent. This is a
smoke test, not a benchmark score.

Speed

All numbers are for one stream on the test machine above. The prompts are real
coding-agent prompts. Each run generates 1,024 tokens with temperature 1.0, top_p
0.95 and top_k 20.

Decode is the median of 4 seeds at 2K and 64K. At 200K it is the total over 4
prompts × 4 seeds, with a warm page cache.

Serving

The results above were measured with this exact SGLang build:

• Commit 93540334d2
  (tag qwen38-fn-nvfp4-2026-10-10
  in beginning-ai/sglang).
• It is upstream SGLang
  f3fe753475 plus 9
  commits
  (compare).
  The censored model card
  lists what each commit adds.

pip install -e "git+https://github.com/beginning-ai/sglang@93540334d2ce687f0677bebb42dbd252b218e92f#egg=sglang&subdirectory=python"

We have not tested this checkpoint on unpatched SGLang.

python3 -m sglang.launch_server \
  --model-path beginning-ai/Qwen3.8-Flash-Next-Uncensored-NVFP4 \
  --trust-remote-code \
  --quantization modelopt_fp4 \
  --dtype bfloat16 \
  --kv-cache-dtype fp8_e4m3 \
  --page-size 64 \
  --context-length 262144 \
  --linear-attn-decode-backend flashinfer \
  --linear-attn-prefill-backend flashinfer \
  --mamba-radix-cache-strategy extra_buffer \
  --speculative-algorithm NEXTN \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Notes:

• PLE table on NVMe. If the 47.7 GiB PLE table does not fit in host RAM, as on
  our 64 GB machine, add --ple-offload-backend nvme (patched SGLang).
• 524K context. Add --context-length 524288 and set
  SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1. Also pass
  --json-model-override-args with a YaRN rope_parameters block: rope_type: "yarn", factor: 2.0, original_max_position_embeddings: 262144, and the base
  config's mrope_* / rope_theta / partial_rotary_factor values.
• Decode kernels. SGLANG_FP8_ROW_WEIGHTS=1 (FP8 output head and FP8 draft
  projections, converted at load time) and SGLANG_NVFP4_DECODE_MOE=1 turn on the
  patched decode kernels.

Files

License and credits

• License: the Qwen Community License 1.0, the same as the base model (see
  LICENSE).
• Source model: orcarouter/Qwen3.8-Flash-Next-Uncensored
  by OrcaRouter.
• Base model: Qwen/Qwen3.8-Flash-Next
  by the Qwen team. The PLE table is from
  Qwen/Qwen3.8-Flash-Next-FP8.
• Quantization: NVIDIA TensorRT Model Optimizer (ModelOpt).