license: other
license_name: qwen
base_model: lovedheart/Qwen3.8-Flash-Next-NVFP4-FP8
base_model_relation: quantized
pipeline_tag: text-generation
library_name: sglang
tags:

• abliterated
• uncensored
• nvfp4
• fp8
• modelopt
• sglang
• moe

Qwen3.8-Flash-Next — Abliterated, NVFP4 experts + FP8_PB_WO dense

An abliterated build of Qwen3.8-Flash-Next in the modelopt MIXED_PRECISION format:
NVFP4 group-16 experts with FP8_PB_WO (128x128 block, weight-only) dense layers, and
the PLE n-gram table in FP8.

This combination had no abliterated build. Every published one is uniform NVFP4 or
compressed-tensors, both of which lose the block-scaled FP8 dense path — which on
sm120 is what the KV pool depends on. This exists to fill that gap.

It is a transplant, not a new abliteration. The refusal direction was recovered
from published weight deltas of an existing abliterated checkpoint and re-applied to
this quantization. See METHOD.md.

Attribution

The credit for the ablation belongs upstream. What is new here is the transplant into
a different quantization, and the measurement that says it worked.

What was changed

W' = W - 1.5013 * r (r^T W) applied to 24,676 residual-writing tensors:

alpha = 1.5013 was measured from the reference build (sd 0.0004 across 12
tensors), not assumed — it is a deliberate over-projection, not a plain rank-1
projection. Attention q/k/v, expert gate_proj/up_proj, norms and the vision
tower are untouched.

Evaluation

GSM8K, 1319 problems, temperature 0.6, top_p 0.95, max_tokens 8192, identical
harness and settings for both:

No measurable capability cost. Refusal behaviour does change on prompts the base
declines. Multimodal transcription is unchanged (same image-token count, same output
structure, 4.4 s vs 4.5 s).

Serving

Built and tested on SGLang, one RTX PRO 6000 Blackwell (sm120, 96 GB):

--quantization modelopt --kv-cache-dtype fp8_e4m3 --ple-offload-embedding
--mem-fraction-static 0.975 --context-length 850000
--speculative-algorithm NEXTN --speculative-num-steps 3

Serving profile is identical to the unmodified quantization: KV pool 853,056 tokens,
850k context, 4.07 GB free VRAM. --ple-offload-embedding is required — the n-gram
table is ~47.7 GiB and lives in host RAM.

Note reasoning_effort matters a lot on this model. Without it, it will spend an
entire token budget thinking and return an empty response; "reasoning_effort": "medium" in chat_template_kwargs fixes it.

Caveats

• The quality of the ablation is inherited from the upstream build.
• Single-direction abliteration is contested — see
  There Is More to Refusal in LLMs than a Single Direction.
• GSM8K is one benchmark. Unchanged arithmetic does not prove unchanged everything.
• Safety alignment has been substantially removed. It will answer things the original
  refuses. Released for research and for local use on hardware you own.

License follows the base model (other / Qwen terms), as with every checkpoint in the
chain above.