license: other
license_name: qwen-community-license-1.0
license_link: LICENSE
base_model:

• windowsxp811203/Qwen3.8-Flash-Next-Abliterated-NVFP4

• doth4580/Qwen3.8-Flash-Next-DotH-MTP-BF16
  base_model_relation: finetune
  pipeline_tag: image-text-to-text
  library_name: vllm
  tags:

• nvfp4

• fp4

• modelopt

• fp8

• vllm

• abliterated

• uncensored

• qwen4_exp

• qwen3.8

• flash-next

• moe

• mtp

• speculative-decoding
  language:

• en

• zh
  extra_gated_heading: "Access requires acknowledging what this model is"
  extra_gated_description: >-
  This is a refusal-ablated ("abliterated") model. It complies with requests that
  the base model refuses. Access is granted automatically once you share your
  contact details and accept the terms below.
  extra_gated_prompt: >-
  Terms of use

  1. This model has had its learned refusal direction removed. It will not reliably
     decline harmful requests. Any safety behaviour must be supplied by your own stack.

  2. You are solely responsible for how you deploy, serve, or redistribute it, and
     for complying with all laws and regulations that apply to you.

  3. If you expose it to third parties, you agree to place your own moderation and
     policy layer in front of it. The model will not do that job for you.

  4. It is provided as-is, without warranty of any kind. The authors accept no
     liability for any use or consequence.

  5. Licensed under the base model's license (see the license field above);
     attribution to the base model is expected in derivative work.

  By requesting access you agree to share your Hugging Face username and email
  address with the repository owner.
  extra_gated_fields:
  Full name: text
  Organization or affiliation: text
  I will put my own moderation layer in front of this model before exposing it to third parties: checkbox
  I accept the terms above and take full responsibility for my use of this model: checkbox
  extra_gated_button_content: "I understand. Request access."

Qwen3.8-Flash-Next-Abliterated — NVFP4 with a tuned MTP head

This is windowsxp811203/Qwen3.8-Flash-Next-Abliterated-NVFP4
(v2, revision 5a81348) with one thing changed: the dense part of its multi-token-prediction (MTP) head, the small
drafter that vLLM's speculative decoding uses. Every other weight is byte-for-byte the public checkpoint's.

The head is built on doth4580's
DotH MTP head (revision 93ee3d0b, Qwen
Community License 1.0), a retraining of Flash-Next's 26 dense MTP tensors on the model's own outputs. Most of the speed-up
on this card comes from DotH: on the same benchmark, DotH's head (with the edit described under Lineage), served
the same way (k=6, FP8 draft head), already beat the stock head on every suite, and our further training adds about 2.7 % on Chinese chat and 0.5 % elsewhere on top of
it. Thank you, doth4580.

Speculative decoding keeps the target's output distribution: the full model verifies every drafted token, so the head
is there to change speed, not the model's answers or its refusal behaviour. On this stack greedy outputs are not
bit-reproducible, though (see Pre-registered verdict): against stock MTP k=4 in the same session, this head's greedy
outputs (32 prompts × 256 tokens) diverged beyond a near-tie on 15 prompts, no more often than two restarts of stock
MTP k=4 diverged from each other in an earlier session (19 of 32).

Measured result (one H200, vLLM 0.32.0 with the two patches in patches/, greedy, one request at a time; details
under Results): with num_speculative_tokens=6 and an FP8 copy of the LM head for the drafter, this
checkpoint decoded faster than the public checkpoint's own MTP head at that head's best setting (k = 3 … 7, chosen per
suite) on all 13 suites: +9.8 % (geometric mean), worst suite +5.5 %. Against k=3, the setting the public card
recommended until now: Chinese chat +10 %, the other suites +32 %.

What differs from the public NVFP4 repo

The 26 tensors (88,929,792 parameters), all under mtp.: fc_embedding, fc_hidden, pre_fc_norm_embedding,
pre_fc_norm_hidden, hyper_connection_mixer.{hc_norm, input_mix_weight_down, input_mix_weight_up},
layers.0.self_attn.{q_proj, k_proj, v_proj, o_proj, q_norm, k_norm},
layers.0.{attn,mlp}_hyper_connection.{block_inject_weight, hc_norm, input_mix_weight_down, input_mix_weight_up},
layers.0.mlp.gate (the router), layers.0.mlp.shared_expert.{gate_proj, up_proj, down_proj},
layers.0.mlp.shared_expert_gate. This is exactly the set DotH retrains.

Size, memory and everything about the main model are as on the public card: 125.9 GiB on disk; with MTP on, vLLM 0.32
puts 78.7 GiB of weights on the GPU and keeps the 47.7 GiB FP8 n-gram table in pinned host RAM. The FP8 draft LM head
adds a private ~0.64 GB copy of the LM head.

Running it

Every number on this card was measured on vLLM 0.32.0 with two patches, both in patches/:

Both were checked against a clean vLLM 0.32.0 install: applied to the untouched files they reproduce, byte for byte, the
files our benchmark server ran. Neither was tested on any other vLLM version.

pip install vllm==0.32.0
SP=$(python -c "import importlib.util, os; print(os.path.dirname(os.path.dirname(importlib.util.find_spec('vllm').origin)))")
hf download windowsxp811203/Qwen3.8-Flash-Next-Abliterated-NVFP4-TunedMTP --include "patches/*" --local-dir tunedmtp
P=$PWD/tunedmtp/patches
(cd $SP && patch -p1 < $P/qsa-ring-widen-PR54912.diff)   # required for k >= 5
python $P/patch_vllm_draft_lmhead.py $SP/vllm             # optional: FP8 draft LM head (keeps *.orig backups)

VLLM_USE_V2_MODEL_RUNNER=1 VLLM_USE_FLASHINFER_SAMPLER=0 \
vllm serve windowsxp811203/Qwen3.8-Flash-Next-Abliterated-NVFP4-TunedMTP \
  --language-model-only --max-model-len 32768 --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 6, "draft_lm_head_quantization": "fp8"}'

• Without the FP8 head patch, drop the draft_lm_head_quantization key. k=6 still works; on the stock head the FP8
  copy made decoding about 5.5 % faster (about 5 % less time per token) at the same k, with about the same acceptance
  length (geometric mean −0.3 %; −4.4 … +2.9 % per suite) (13 suites, session D); expect
  a similar slowdown here without it (not measured with this head).
• k=6 is the setting we tested and recommend. k=5 is not recommended: it failed our pre-registered check (below).
  Other values of k were not measured with this head.
• VLLM_USE_V2_MODEL_RUNNER=1 is required for the FP8 draft head, which also needs an FP8-capable GPU (sm_89 or
  newer). The other settings are the public card's, for the same reasons: on our H200s vLLM 0.32.0's HUMMING NVFP4-MoE kernel failed to JIT, so we emptied
  _HUMMING_PREFERRED_CAPABILITIES in vllm/model_executor/layers/quantization/utils/humming/priority.py to use
  Marlin, and VLLM_USE_FLASHINFER_SAMPLER=0 avoided a FlashInfer sampler JIT failure. Our benchmark servers also passed
  --max-num-seqs 32 --gpu-memory-utilization 0.90 --enable-auto-tool-choice --tool-call-parser qwen3_coder, used
  --max-model-len 262144 for the long-context runs, and also carried patches from our DSpark/DFlash experiments
  (auxiliary hidden-state capture, the drafter allow-list, a torch top-k fallback), none of which is on the MTP path.
• Thinking is on by default ("chat_template_kwargs": {"enable_thinking": false} turns it off per request).
• Only vLLM, text-only (--language-model-only), was tested.

Results

Setup. One H200 (sm_90), one vLLM 0.32.0 benchmark server at a time (--tensor-parallel-size 1), NVFP4 experts on
Marlin; greedy decoding (temperature 0), one request at a time; TPOT = median time per output
token over a suite; AL = acceptance length (tokens committed per verification step, including the bonus token).
The HumanEval, MATH-500, MT-Bench-zh and WildChat thinking suites use medium reasoning effort and up to 2048 new
tokens; the 8K agent suite uses xhigh effort and up to 1024; the long-context agent cuts use xhigh effort (44 of 45
prompts; one medium) and up to 512; non-thinking suites up to 512. Prompts:
MT-Bench-zh (80), WildChat Chinese (100), WildChat Traditional Chinese (50, held out from training), and
slices of a 155-prompt English/code/math/agent mix: HumanEval, GSM8K, MATH-500, MT-Bench, and 8K-token agent prompts
with tools cut from held-out Open-SWE-Traces sessions.

How the comparison works. Our final session (S3, 2026-11-05/06) ran this head and the DotH-b head (see Lineage)
at k=6 with the FP8 draft head, twice each (DotH-b first, then this head first), plus stock-head MTP k=4 at the start
and the end as an anchor for drift (−0.32 %). The stock head at k = 3, 4, 5, 6, 7 was measured in an earlier session (F) that also ran
the DotH-b head, so the margins against the stock head are bridged through DotH-b:
TPOT_F(stock, best k) / TPOT_F(DotH-b) × TPOT_S3(DotH-b) / TPOT_S3(this head) − 1, each session drift-corrected by
its own anchors (both within 1 %). Positive = this checkpoint is faster. S3 ran with no other job of ours on the host
(a shared machine); a short GPU job of ours overlapped the first hour of session F.

TPOT / AL of this head are the mean of its two S3 runs. The stock head's "best k" is the fastest of k = 3 … 7 in session
F for that suite; the "vs k=3" column uses session F's raw k=3 / best-k ratio on top of the bridged margin. For scale:
plain decoding without speculation was 7.84 ms per token on the 155-prompt mix (public card).

Long context (agent prompts of 32K, 64K and 128K tokens with tools, 15 each, thinking, --max-model-len 262144):
this head at k=6 + FP8 head ran at 3.24 ms median TPOT (AL 4.09) vs 3.33 ms (AL 4.10) for the DotH-b head in the same
session; per cut (32K / 64K / 128K) −6.2 % / +1.3 % / −1.1 % time per token vs DotH-b (negative = faster).
Bridged to session F and measured against each cut's own fastest stock-head k there (32K: k=3, 3.48 ms; 64K: k=4,
3.42 ms; 128K: k=5, 3.30 ms), the margins over the stock head are +9.7 % / +4.8 % / +9.4 % (32K / 64K / 128K). (Pooled
over all cuts, the stock head's best in F was k=5 at 3.37 ms; k=3: 3.65 ms.)

Pre-registered verdict. The pass rules were fixed before the session ran:

The greedy check exists because on this stack the target itself is not bit-reproducible: a different verify width or a
server restart changes floating-point reduction order, so long greedy outputs drift at near-tied tokens. That is why the
bar is the stock configuration's own restart-to-restart divergence, not zero.

Secondary analyses were also registered before the agent and long-context rows of S3 were looked at. The one with a
pass rule passes: the noisy agent suite pooled over 5 interleaved pairs is +2.9 % vs DotH-b (+7.6 % vs the stock head's
best), giving a secondary k=6 verdict of PASS. The two information-only analyses agree: token-weighted decode time gives
+2.75 % (Chinese) / +0.85 % (other) vs DotH-b; long context, all 45 requests pooled, −2.6 % time per token vs DotH-b
with the same acceptance length (−0.3 %).

Earlier sessions (D and F, 2026-11-04/05), before this head existed: the DotH-b head with the FP8 draft head at k=6
beat the stock head at its best k (3 … 7) in all 18 comparisons (17 suites, some of them overlapping, plus pooled long
context at +4.5 % median), geometric mean +8.0 %, worst +2.5 %. In the same sessions the FP8 draft head alone (stock
head) made decoding about 5.5 % faster at the same k without changing the mean acceptance length, and neither the FP8 head nor the DotH head alone beat the stock head on every
suite.

Limitations

• k=6 only. k=5 failed its check and other k were not measured with this head.
• One GPU type, one request, greedy. Everything was measured on one H200 per server, single stream, temperature 0,
  text-only. Concurrency, sampling (temperature > 0), tensor parallelism, other GPUs (including Blackwell and its
  native FP4 path) and other engines were not measured with this head.
• Patched vLLM. k ≥ 5 needs the QSA ring backport on 0.32.0 (and, by our code reading, on 0.33.0); the FP8 draft head
  needs the #59973 backport on 0.32.0. vLLM 0.33+ was not run.
• Small, noisy suites. The code, math and agent rows have 10–20 prompts. The 8K agent suite (15 requests) is the
  noisiest: its median TPOT differed by 14–18 % between identical runs of the same configuration, which is why its
  pooled 5-pair result is reported next to the 2-pair one.
• The first head failed. Our pilot head (m11p, the step-182 export of a 242-step pilot run) failed the same
  pre-registered rules in session S2: at k=6 its agent suite was 9.9 % slower than DotH-b (limit −4 %) and 5.8 % slower
  than the stock head at its best k (the first rule), and k=5 failed as well. A later analysis attributed that to the suite's
  noise, but the rule was not changed after the fact; instead we trained this second, longer run, registered the
  secondary analyses above before seeing its agent rows, and released only this head.
• Checkpoint choice. The selection rule used for the pilot (least-moved checkpoint within 0.3 points of the best
  offline score) picked step 300 of a different, shorter run (offline score +2.42 %), because it was inside that band
  and had moved less. We overrode the rule and took the final checkpoint of the longer run (step 1,292), which had the
  best offline score (+2.61 %, i.e. 0.19 points above the rule's pick, inside the band). This was a post-hoc choice,
  made by us after seeing the scores; the speed rules it was then judged by in S3 were fixed before S3 ran.
• Modest gain over DotH. Against the DotH-b head the improvement is +2.7 % on Chinese chat and about +0.5 %
  elsewhere; the bulk of the margin over the stock head comes from DotH's head, the FP8 draft head and k=6.

Lineage and training

1. Stock head: Qwen's MTP head as shipped in the abliterated checkpoint (our abliteration edits five of its tensors:
   the four dense writers named under DotH-b below plus the routed experts' fused down_proj; see the
   parent card). This head keeps that fifth
   tensor exactly as it is in the abliterated checkpoint (the routed experts are not retrained).
2. DotH (doth4580/Qwen3.8-Flash-Next-DotH-MTP-BF16
   @ 93ee3d0b): doth4580 retrained the 26 dense tensors on Flash-Next's own outputs (forward KL to the model's
   truncated next-token distribution along a 10-step draft chain; see their card). We used only its dense bf16 file.
3. DotH-b: DotH's tensors placed into this checkpoint, with the four tensors our abliteration edits in the stock head
   (fc_embedding, fc_hidden, layers.0.self_attn.o_proj, layers.0.mlp.shared_expert.down_proj) projected exactly
   as the abliteration did (the refusal direction of layer 37, λ = 1.5); everything else as in NVFP4-v2.
4. This head (m11c): DotH-b trained further by us; the export is stitched in as trained (no further projection).

Recipe. Only the 26 dense tensors train; the head's 512 routed experts, its indexer, and the whole main model are
frozen. The teacher is the abliterated NVFP4-v2 target itself: its hidden states (the four hyper-connection streams
after layer 47, as the MTP head reads them when serving) and its next-token distribution (its LM head on the final
mixer output, truncated as vLLM samples at the default T=1 / top-k 20 / top-p 0.95) are produced on the fly by vLLM 0.32
extraction servers and never stored. Loss per position along a 7-step draft chain, each step feeding its own output
back and attending to its own chain's earlier steps as in serving: KL(target ‖ head) + 0.2 × cross-entropy to the
target's greedy token, with DotH's step weights 1, 0.7, 0.5, 0.35, 0.25, 0.18, 0.12. AdamW (β 0.9/0.95, no weight
decay), peak learning rate 3e-5, 20 warm-up steps, gradient clip 1.0, 65,536 anchor positions per step (at most 8,192
per sequence), sequences up to 32K tokens. The run first trained 450 steps on a cosine schedule cut short to 450 steps,
then resumed from that checkpoint (optimizer state included) on the full 1,292-step plan's cosine from step 451
(2.2e-5 → 0). About 8.5 hours on one H200 trainer fed by two H200 extraction servers.

Data. The training text only supplies contexts; every target is the model's own distribution at that position.

No dedicated harmful-prompt set was used. (Real-user sources such as WildChat naturally contain some harmful
requests; no set built for that purpose was added.) The private harmful-prompt set used elsewhere in this project was excluded: the
Chinese part was drawn only from the sources listed, and every row of the general mix matches a row of the regenerated
public-prompt files, which contain 0 rows of that set. Before training, no training prompt shared its opening with any of 971 evaluation prompts.

License and attribution

The weights are a derivative of Qwen3.8-Flash-Next and are released under the
Qwen Community License 1.0 — not Apache-2.0 — the base model's license, which the DotH head also carries.
Publishing and redistributing derivative weights is permitted. Running a Model-as-a-Service or AI Work Assistant
business requires a separate license from Qwen before any commercial use; internal use is exempt provided the
model, its outputs and its capabilities reach no third party.

The files in patches/ are not weights and are not under the Qwen license: they contain code derived from vLLM
(backports of PRs #54912 and #59973), Copyright contributors to the vLLM project, licensed under the Apache License 2.0
(full text in patches/LICENSE-vLLM-Apache-2.0.txt). Each file states that it was modified; our changes are released
under the same Apache-2.0 license.

Credits: Qwen (Qwen3.8-Flash-Next and its MTP head, Qwen/Qwen3.8-Flash-Next-FP8's n-gram table, via the public
checkpoint); doth4580 (the DotH MTP head this head starts from);
gorbatjovy (the first loadable NVFP4 build of this model, credited on the public
card); the vLLM project (PRs #54912 and #59973, backported here).

Training data (prompts and contexts only; no dataset answer was used as a target):

Contains information from WildChat-1M (https://huggingface.co/datasets/allenai/WildChat-1M) and WildChat-4.8M
(https://huggingface.co/datasets/allenai/WildChat-4.8M), made available under the ODC Attribution License
(https://opendatacommons.org/licenses/by/1-0/). Zhao, Ren, Hessel, Cardie, Choi, Deng. "WildChat: 1M ChatGPT
Interaction Logs in the Wild." ICLR 2024. Changes: conversations filtered (length/URL-only/exact and
near-duplicate/template/per-IP caps), capped at 6 user turns, final assistant answer removed.

Prompts from OpenCodeInstruct by NVIDIA (https://huggingface.co/datasets/nvidia/OpenCodeInstruct), CC BY 4.0
(https://creativecommons.org/licenses/by/4.0/). Ahmad, Ficek, Samadi, Huang, Noroozi, Majumdar, Ginsburg.
"OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMs." arXiv:2504.04030 (2025). Changes: random
sample of shard 00000, prompts only.

Agent trajectories from Open-SWE-Traces by NVIDIA (https://huggingface.co/datasets/nvidia/Open-SWE-Traces), CC BY 4.0.
Ahmad, Ludwig, Majumdar, Ginsburg. "OPEN-SWE-TRACES: Advancing Dual-Mode Multilingual Distillation for Software
Engineering Agents." Findings of EMNLP 2026. Changes: subset of sessions; tool definitions converted to OpenAI
function format; per-message null fields normalised; sessions cut at a turn boundary.

Permissive sources (Apache-2.0 / MIT): mlabonne/open-perfectblend (Apache-2.0), Congliu/Chinese-DeepSeek-R1-Distill-data-110k
(Apache-2.0), twinkle-ai/tw-reasoning-instruct-50k (MIT), lchakkei/OpenOrca-Traditional-Chinese (MIT),
Heng666/Traditional_Chinese-aya_dataset (Apache-2.0), meta-math/GSM8K_zh train split (MIT), LooksJuicy/ruozhiba
(Apache-2.0), ise-uiuc/Magicoder-OSS-Instruct-75K (MIT), project-numina/NuminaMath-CoT (Apache-2.0), Team-ACE/ToolACE
(Apache-2.0), NousResearch/hermes-function-calling-v1 (Apache-2.0), glaiveai/glaive-function-calling-v2 (Apache-2.0).
Evaluation prompts (never trained on): MT-Bench, MT-Bench-ZH, MATH-500, GSM8K test, HumanEval, and held-out WildChat /
Open-SWE-Traces items.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time.
如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

This model will not refuse. It is published for alignment and safety research. You are responsible
for your use of it and for complying with applicable law.