license: apache-2.0
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: vllm
tags:

• qwen
• qwen3.8
• flash-next
• qwen4
• moe
• nvfp4
• fp4
• fp8
• quantized
• abliterated
• uncensored
• vllm
• dgx-spark
• vision-language
• function-calling
• reasoning
• ai-red-team
• red-teaming

Qwen3.8-Flash-Next-Uncensored-NVFP4-FP8PLE

NVFP4 abliterated build of Qwen/Qwen3.8-Flash-Next in which the PLE n-gram embedding table is quantized to FP8 e4m3 (single global scale) instead of bf16. The result is byte-for-byte the same model as orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 with one structural change: the 102.4 GB bf16 PLE shard is replaced by 10 FP8 shards, shrinking the checkpoint from 170.93 GiB to 123.25 GiB on disk. The PLE layout is deliberately matched to RadixArk/Qwen3.8-Flash-Next-NVFP4 so the checkpoint slots into the MiaAI-Lab Dual-DGX-Sparks recipe unchanged.

Provenance chain:

Qwen/Qwen3.8-Flash-Next (BF16)
  └─ orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4   (abliteration + NVFP4 experts / FP8 attention; PLE kept bf16; 170.93 GiB)
       └─ this repo                                    (PLE n-gram table → FP8 e4m3, global scale; 123.25 GiB)

What changed vs orcarouter (everything else identical)

Note: the two small PLE metadata tensors (ngram_heads_offsets, ngram_heads_vocab_sizes) stay bf16/int in model-00003-of-00017.safetensors, exactly as upstream.

Quantization details: single global scale per table, scale = amax/448 rounded to bf16 (global amax = 0.0893555, scale = 0.0001993179321) — the same law RadixArk published for its FP8 PLE, and this value matches their shipped scale to the displayed digit. Worst-case measured single-value dequantization error: 0.00317 in bf16 units. The table was quantized from orca's own abliterated bf16 PLE table (streamed 1:1, no resampling), not transplanted from any other checkpoint; see Similarity to RadixArk.

Similarity to RadixArk/Qwen3.8-Flash-Next-NVFP4

This checkpoint reproduces RadixArk's FP8-PLE serialization so the two are drop-in interchangeable at the layout level:

What is NOT the same: the PLE weights. RadixArk quantized from the PLE tables shipped in the official Qwen/Qwen3.8-Flash-Next-FP8 revision; this repo quantizes from orca's abliterated bf16 table, and independent byte-comparison shows the PLE value_proj matrices differ (~44% of bytes) between the two sources. Do not mix PLE files across these two checkpoints: the loader layout is compatible, the data is not. Architecture-level config (48 layers, 36 Gated-DeltaNet + 12 full-attention at full_attention_interval: 4, 512 experts, MTP) is the same family in both.

config.json layer_types uses full_attention ×12 / linear_attention ×36 (no qwen_sparse_attention entries), matching RadixArk and required by the ModelConfig allowlist of the pinned vLLM image.

Compatibility with the MiaAI-Lab Dual-DGX-Sparks recipe

Built and verified against MiaAI-Lab/Qwen3.8-Flash-Next-Dual-DGX-Sparks:

• Image: vllm/vllm-openai:qwen38-flash-next (vLLM v0.1.dev20073+g8e685d198) — the recipe's pinned day-0 image.
• PLE patch: the recipe's bind-mounted resolver shim (files/ple_layer_patched.py → vllm/models/qwen3_8_flash_next/nvidia/ple_layer.py) is the same one. Set PLE_QUANT_OVERRIDE=fp8 (exactly as the recipe's start.sh does for RadixArk): this routes the PLE embedding to the image's Qwen3_8FlashNextPLEFp8EmbeddingMethod, which handles "FP8 shards + one global weight_scale" — this layout. Without it, the engine tries to build a ~102 GB BF16 PLE and OOMs on boot.
• Launch flags: the recipe's full flag set applies unchanged (--tensor-parallel-size 2 --nnodes 2 --enable-expert-parallel --all2all-backend allgather_reducescatter --speculative-config '{"method":"mtp","num_speculative_tokens":3}' --gpu-memory-utilization 0.835 --max-num-seqs 8 --max-num-batched-tokens 8192 --kv-cache-dtype auto --load-format safetensors --safetensors-load-strategy lazy --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --compilation-config '{"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY"}' plus the YaRN --hf-overrides for 1M context and VLLM_ALLOW_LONG_MAX_MODEL_LEN=1).
• Pointing it at this repo: in the fork's .env, set MODEL_ID=lychee888/Qwen3.8-Flash-Next-Uncensored-NVFP4-FP8PLE (the recipe's start.sh downloads from HF and rsyncs to the worker). Alternatively, stage the files locally on both nodes and use a local-directory MODEL_ID (the maintained fork of the recipe supports absolute-path MODEL_ID).
• KV cache: the recipe mandates KV_CACHE_DTYPE=auto (bf16). Do not set FP8 KV: on sm_121 it crashes QSA prefill in this image (documented by the recipe and confirmed by multiple deployments).

End-user notes (read before deploying)

1. Smaller PLE footprint, slightly tighter KV pool than RadixArk. The FP8 PLE halves the runtime embedding memory vs the bf16-PLE upstream (102.4 GB → 51.2 GB on disk). Against RadixArk specifically, this PLE carries ~3.5 GiB of more data (51.20 vs 47.68 GiB), so in an otherwise identical launch the KV pool lands slightly below the reference build's measured 2,481,424 tokens (35.11 GiB at GPU_MEMORY_UTILIZATION=0.835, 1M YaRN, MTP3, as measured in the recipe repo). The recipe's own measured ladder for that configuration: 0.835 → 2.48M tokens; 0.85 → 2.57M; --kv-cache-memory=45502283776 (42.38 GiB) → ≈3.0M. No 2.7M pool figure is documented in the recipe or in either upstream repo; treat any such number as unverified. Measure on your own boot and treat that log as ground truth:

   docker logs vllm-fn 2>&1 | grep -E "Available KV cache memory|GPU KV cache size"
   

2. PLE_QUANT_OVERRIDE=fp8 is mandatory on the pinned image (see above). The config.json ple_embedding_dtype field is informational for this image's loader and runtime-neutral.
3. Boot gotchas already fixed in this checkpoint: (a) layer_types contains only full_attention/linear_attention, which the pinned image's ModelConfig allowlist accepts; (b) n-gram shard keys are single-level ngram_embedding.shard_N.weight, which is what the image's Qwen3_8FlashNextNGramEmbedding loader expects. If you regenerate the FP8 PLE yourself, keep both invariants.
4. Accuracy impact of FP8 PLE: bounded by one E4M3 ULP per element on the n-gram prefix-lookup table. The PLE table is not a refusal/behavior-carrying tensor in this model and abliterated behavior is unchanged relative to the bf16-PLE upstream; no eval deltas were observed in smoke testing, but treat any PLE-quantization-sensitive task (heavy prefix/n-gram retrieval) as something to spot-check for your use case.
5. Hardware: Blackwell with FP4 tensor cores (DGX Spark GB10, B100/B200/GB200, RTX 50-series) for the NVFP4 experts; qwen4_exp runtime required (the pinned image above).
6. Check integrity on download: every file is listed in SHA256SUMS.txt (39 entries); verify with sha256sum -c SHA256SUMS.txt.

Requirements

• vllm/vllm-openai:qwen38-flash-next (or any build with qwen4_exp + the PLE FP8 resolver shim).
• transformers>=5.16 for config/tokenizer handling if loading outside vLLM.
• ~126 GiB of local storage per node when using the dual-Spark recipe.

Upstream provenance & license

Derived from orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4 (Apache 2.0), itself an abliteration + NVFP4 quantization of Qwen/Qwen3.8-Flash-Next. License: Apache 2.0 (inherited; see upstream for full text and terms). This model has had its safety alignment substantially removed; it is released strictly for legitimate research, interpretability, AI-safety and red-teaming work. You assume full responsibility for its use; add your own safety and moderation layers before any deployment. The recipe.yaml included here is the upstream ModelOpt quantization recipe, carried over for provenance.