license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model: local-inference-lab/Qwen3.8-Flash-Next-NVFP4
base_model_relation: finetune
pipeline_tag: image-text-to-text
tags:

• nvfp4
• mxfp8
• modelopt
• lossless-compression
• csf
• abliterated
• uncensored

Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated

Reduced-refusal model. A refusal-direction edit was applied on purpose.
Read the Safety section before you deploy this.

An NVFP4-CSF container of an abliterated Qwen3.8-Flash-Next. The
routed-expert block-scale planes are stored in the compressed CSF form. Every
weight nibble and every other tensor is unchanged from the build described
below, and all tensor names, dtypes and shapes are preserved.

This repository replaces an earlier release. The same weights were first
published as huginnfork/Qwen3.8-Flash-Next-NVFP4-Abliterated tag v4, in the
plain uncompressed layout: 86 files, 40 safetensors shards, 106.41 GB. That tag
has been removed, and this repository is its sole successor. The weights are
bit-identical; only the scale encoding differs, which makes this container
3.58 GB smaller and readable by the NVFP4-CSF path. Every number reported below
was measured on that build, and the raw result files travel with this repository
under measurements/.

Provenance

The container format is NVIDIA-ModelOpt mixed precision with CSF scale
encoding, designed and published by Local Inference Lab, Inc. We encoded these
planes ourselves from our own weights. No compressed bytes were copied from any
Local Inference Lab repository, and this repository is not a mirror of one.

What the abliteration did

Applied in place on the quantised grid — no dequantise, no re-quantise. A
rank-1 refusal direction, biprojected against the harmless mean, captured at
layer 46 and applied with alpha = 1.0. 25,188 tensors were edited:

No scale tensor was edited. No router, no gate_proj or up_proj, no
q/k/v_proj, no vision tensor, no lm_head, no PLE table entry.

Editing a value that already sits exactly on the quantisation grid attenuates
the edit under plain round-to-nearest, so a constrained requantiser chooses codes
that both approximate the edited weight and hold v·W = 0 per column within a
±1 ULP budget. Measured residual leakage is about 3.5e-06 on NVFP4 modules
and 4.3e-07 on MXFP8 modules.

Measurements

Measured on the predecessor build, whose weights are bit-identical to the ones
here. Nothing was re-measured on this container. Raw result files are in
measurements/.

Capability

On lavd neither arm produced a wrong answer — every miss on both sides is a
40k-token cap hit. The GPQA difference is not significant (p = 0.286).

Refusal, held-out rows, and the regime matters enormously

A refusal number is a protocol and scorer pair, and the protocol dominates.
The same weights move by a factor of about 19 between the two regimes below. Do
not quote one cell on its own.

Counter-intuitively thinking-off is the harder regime: the unedited parent
refuses 100 % there against 89.7 % with thinking on. Protocol A is the
model-forge gate, which renders its prompts with enable_thinking=False and
scores them the same way. The direction used here was captured in that same
regime, which is why the two are self-consistent.

Over the full 200-row harmful set, protocol A gives 0/200 under the model-forge
scorer and 6/200 under our stricter 47-marker scorer. Swapping the scorer moves
the number a few points; swapping the thinking regime moves it ~70.

The CSF container

Only the main-decoder routed-expert block scales are compressed: 73,728
planes, which is 48 layers × 512 experts × 3 projections. Each logical
weight_scale becomes two ordinary safetensors tensors.

MTP experts, attention, shared experts, the PLE n-gram table, every global scale
and every activation scale keep a plain weight_scale. config.json declares
weight_scale_encoding: "csf" on the 48 routed-expert recipes and on nothing
else.

Verification

Built and checked with remote/csf_codec.py, remote/build_csf_container.py
and remote/verify_csf_container.py in our pipeline repository. Three checks,
all exhaustive rather than sampled:

1. All 73,728 planes decode back to the source bytes exactly. Verified by
   reading the written shards, not from build-time state.
2. All 227,865 other tensors are byte-identical to v4, with dtype and
   shape unchanged. The 149,093 FP32 tensors — which include the trained
   normalisation parameters — are preserved verbatim.
3. The index maps every tensor, and weight_scale_encoding appears on
   routed-expert recipes only.

The encoder was additionally validated against a published CSF plane rather
than only against our own decoder. Encoding v4's plain
…layers.0.mlp.experts.0.down_proj.weight_scale reproduces the corresponding
nvfp4_csf_fixed and nvfp4_csf_exceptions tensors of
local-inference-lab/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD byte for byte,
including all 226 exception words. Our independently computed totals also match
that container's own manifest exactly on compressed-plane count, bytes saved,
container byte size and tensor count.

Serving

Needs a vLLM build with the NVFP4-CSF reader and the b12x SM120 kernels.

VLLM_PLE_TABLE_MEMORY=disk vllm serve <path to this checkpoint> \
  --quantization modelopt_mixed --load-format safetensors \
  --tensor-parallel-size 1 --pipeline-parallel-size 1 --dtype bfloat16 \
  --moe-backend b12x --linear-backend b12x \
  --mm-encoder-tp-mode data --mamba-cache-mode align \
  --enable-chunked-prefill --block-size 16

--quantization modelopt_mixed is auto-detected from config.json and may be
omitted. TP1, TP2 and TP4 all satisfy the scale-alignment requirement. Pipeline
parallelism must be 1. Expert parallelism, data parallelism and ubatching are
not supported on the CSF path.

Known dependency gap. The CSF scale loader imports b12x.loader._csf_blocks
and b12x.loader._csf_source. Those two modules are not in any released
package, so this path currently needs a source build of
local-inference-lab/flashinfer. If that is impractical, decode the planes back
to plain weight_scale tensors with our remote/csf_codec.py and drop
weight_scale_encoding from config.json — that yields a checkpoint which runs
on any ModelOpt NVFP4 build, at a cost of 3.58 GB.

These files cannot be loaded by Flare. Its runtime accepts only its own
prepared package format.

Differences from Local Inference Lab's CSF-QAD container

Do not treat the two as interchangeable.

• The vision tower here is BF16. Their main revision additionally
  converted 110 vision linears to MXFP8 and NVFP4. This container inherits the
  unconverted vision weights of the 6909a5b parent, because our abliteration
  never touched vision and we did not reproduce their conversion.
• 40 shards, keeping the parent's shard names, where their main revision
  reshards to 21.
• No manifest.json, build-contract.json, receipts/ or verification.json.
  Those are their build-provenance files and their verify and restore
  subcommands do not apply to this artefact.

Not verified

• GPU inference has not been run on this container. The weights are
  bit-identical to v4, which was served and evaluated, and the compression is
  verified lossless. But this exact container has not been loaded by a runtime.
• No vision evaluation was run on any of our builds.

Safety

This model has had refusal behaviour deliberately reduced. It will answer
requests that the parent declines. Do not deploy it as a general-purpose
assistant without your own safeguards, and do not treat the refusal numbers
above as a safety property.

The refusal numbers above are reported for both thinking regimes on purpose,
because they differ by roughly a factor of 19 on the same weights. Compare them
only against the parent measured under the identical protocol and scorer.

Licence and credits

Qwen Community License 1.0, inherited from the parent. See LICENSE.

Condition 1 of that licence requires the notice to travel with every copy. If
you operate a Model-as-a-Service or AI-assistant business, condition 2 requires
a separate licence from Qwen before any commercial use.

• Qwen/Qwen3.8-Flash-Next — base model, by Qwen.
• local-inference-lab/Qwen3.8-Flash-Next-NVFP4 — the QAD NVFP4 and MXFP8
  quantisation this builds on, by Local Inference Lab, Inc.
• The NVFP4-CSF container format and the b12x kernels — by Local Inference
  Lab, Inc. Neither they nor Qwen endorse this model.