license: apache-2.0
base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:
- en
- zh
tags: - fastload
- abliterated
- uncensored
- qwen
- qwen4
- flash-next
- moe
- nvfp4
- compressed-tensors
- vllm
- vision-language
Qwen3.8-Flash-Next-Uncensored-NVFP4 — Fastload
⚡ This is a fastload repack of
orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4The model is unchanged. Every tensor is byte-for-byte identical to the original — same
architecture, same NVFP4 quantization, same weights, same outputs. Only the file layout changed,
so it can actually be loaded from NFS or a RAM-constrained node.Use this instead of the original when loading is your bottleneck. If you have the original
working fine, you don't need this.
The problem this fixes
Upstream puts the entire PLE n-gram embedding table — 128 BF16 tensors of 763 MiB each — into a single
95.37 GiB file, model-00002-of-00017.safetensors. That one file is 55.8% of the checkpoint. It
exceeds available RAM on most nodes, so neither eager loading nor the page cache can ever hold it, and
every restart re-reads all of it from storage.
What changed
| upstream | fastload | |
|---|---|---|
| Largest shard | 95.37 GiB (55.8%) | 4.00 GiB (2.3%) — 23.8× smaller |
| Shards | 18, wildly uneven | 45, uniform |
| Total size | 170.85 GiB | 170.85 GiB (unchanged) |
| Tensor count | 223,046 | 223,046 (unchanged) |
| Weight values | — | byte-identical, SHA-256 verified |
Each shard is now small enough to become page-cache resident, and reads as one sequential stream.
How it was built
- Raw-byte copy. Tensor payloads are copied as bytes and never decoded into a dtype — numpy has no
BF16 or F8_E4M3, and interpreting them is exactly where a repack silently corrupts weights. - Source order preserved. The upstream checkpoint is already layer-monotonic (46 order regressions
across 222,677 tensors, one per layer boundary), so only the shard boundaries were re-cut at 4 GiB.
An earlier version that reordered tensors caused 22× read amplification on ZFS and was discarded. - Independently verified. All 223,046 tensors were re-read from the output files and SHA-256'd
against the source. 0 missing, 0 unexpected, 0 mismatches.
What this does not fix
- Total size is unchanged (170.85 GiB). If you trip vLLM's "checkpoint fits in 90% of available RAM"
gate for auto-prefetch, you still will. That needs the PLE table quantized to FP8. - Tensor count is unchanged (223,046, of which 73,728 are 4-byte NVFP4 global scales). If your client
issues one round-trip per tensor with no readahead, that latency floor survives. That needs the
per-expert tensors fused into vLLM's stacked MoE layout.
This repack makes streaming possible; it does not by itself reduce the number of reads.
Usage
Drop-in identical to upstream — same config, same tokenizer. Requires a Blackwell GPU and a vLLM build
with qwen4_exp support.
vllm serve quocbao747/Qwen3.8-Flash-Next-Uncensored-NVFP4-Fastload \
--tensor-parallel-size 4 --trust-remote-code \
--enable-expert-parallel --enable-auto-tool-choice --tool-call-parser qwen3_coder
⚠️ Disclaimer — inherited from upstream
This model is abliterated: its safety alignment has been substantially removed. It will comply with
harmful, unethical, or illegal requests the original Qwen3.8-Flash-Next would refuse. Released
strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study,
red-teaming, and robustness evaluation. You assume full responsibility for how you use it and
everything it generates; add your own safety and moderation layers before any deployment.
The repack changes no weights, and therefore changes none of this behaviour.
License
Apache 2.0, inherited from Qwen/Qwen3.8-Flash-Next
via orcarouter/Qwen3.8-Flash-Next-Uncensored-NVFP4.