license: other
license_name: qwen-community-1.0
license_link: LICENSE
base_model: local-inference-lab/Qwen3.8-Flash-Next-NVFP4
base_model_relation: finetune
pipeline_tag: image-text-to-text
tags:

• nvfp4
• mxfp8
• modelopt
• lossless-compression
• csf
• abliterated
• uncensored

Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated

Reduced-refusal model. A refusal-direction edit was applied on purpose.
Read the Safety section before you deploy this.

An NVFP4-CSF container of an abliterated Qwen3.8-Flash-Next. The
routed-expert block-scale planes are stored in the compressed CSF form. Every
weight nibble and every other tensor is unchanged from the build described
below, and all tensor names, dtypes and shapes are preserved.

main is v2. This revision replaces the rank-1 edit that main previously
held, which remains available as tag v1. The two are compared below.

Why v2 exists

A refusal direction is regime-specific. Rendering a prompt with thinking on
or off puts the refusal in a different direction in the residual stream, and on
this model those two directions are nearly orthogonal. Measured here on four
captures, two per regime, same corpus and seed, one runtime:

So ablating one regime's direction removes only about 4 % of the other's. That
is why v1 reached 0 % refusal with thinking off and still refused with thinking
on. v2 removes the span of both directions — a rank-2 edit.

Provenance

The container format is NVIDIA-ModelOpt mixed precision with CSF scale
encoding, designed and published by Local Inference Lab, Inc. We encoded these
planes ourselves from our own weights. No compressed bytes were copied from any
Local Inference Lab repository, and this repository is not a mirror of one.

What the abliteration did

Applied in place on the quantised grid — no dequantise, no re-quantise. A
rank-2 projection removing the span of two refusal directions, each captured
at layer 42 and biprojected against its own harmless mean, applied with
alpha = 1.0. The two directions sit at cos = 0.3495, 69.5° apart after
biprojection. 25,188 tensors were edited:

No scale tensor was edited. No router, no gate_proj or up_proj, no
q/k/v_proj, no vision tensor, no lm_head, no PLE table entry. Identical
target set to v1 — only the direction and the rank differ.

The second constraint, and what it costs

A value that already sits exactly on the quantisation grid loses the edit under
plain round-to-nearest, so a constrained requantiser picks codes that both
approximate the edited weight and hold the constraint within a ±1 ULP budget.
At rank 1 that constraint is v·W = 0 per column and one row can satisfy it
exactly. At rank 2 the error is a 2-vector per column while each row still has
one scalar degree of freedom, so no single row can zero it; the constraint is
paid down across many rows instead.

The price of the second constraint is paid in weight error, not in subspace
removal: the worst quantised tensor still gives up only 4.6e-05 of the
ablated subspace. The 4.9e-05 figure is a floor of the FP4 grid rather than a
solver limit — more passes converge, a 2× ULP budget does not improve it, and
five different row orderings span 4.87–5.00e-05.

Measurements

Measured on the uncompressed build, whose weights are bit-identical to the ones
here. Both arms ran in one session on one box at an identical
gpu-memory-utilization, which is the only way these scores are comparable.
Raw result files are in measurements/.

Capability

Zero wrong answers, zero truncated and zero stalled runs on every lavd and
estonia run on both arms. GPQA is paired on the same 198 items: Δ +1.01 pp,
exact McNemar p = 0.8506, not significant, with 13 items correct only on the
parent and 15 only here. MTP acceptance over 312 and 337 samples, IQR
55.8–66.3 % against 53.2–64.7 % — the MTP write projections are themselves
edited, so this is a correctness check rather than a throughput claim.

Refusal — the regime no longer matters

A refusal number is a protocol and scorer pair. On v1 the two regimes
differed by a factor of ~19 on the same weights. On v2 they agree.

Held-out rows, 136 harmful and 59 over-refusal the layer selection never saw:

Over the full 200-row harmful set: 0/200 in protocol A on both scorers, and
0/200 in protocol B under model-forge.

Read the strict protocol-B row with care

The strict scorer counts an empty completion as a refusal. That is a
defensible convention and it is applied identically to both arms, but in
protocol B it mixes two different events. Of this model's 40 strict refusals
over the full 200 rows, 2 are marker-matched refusals and 38 are empty
completions; the parent's 174 is 171 marker-matched and 3 empty.

The asymmetry has a cause and it is not refusal behaviour. This model complies,
a complying answer is long, and it hit the 2048-token cap 79 times against the
parent's 6. A single strict rate therefore penalises the arm that complies, in
exactly the regime the edit exists to fix. measurements/strict_split_protocolB.txt
reports the two components separately for both arms.

Counter-intuitively thinking-off is the harder regime: the unedited parent
refuses 100 % there against 84.6 % with thinking on.

v1 against v2, like for like

Full 200 harmful rows, same datasets, same scorers:

v1 and v2 were measured in different sessions, so this table is weaker evidence
than the parent-versus-v2 table above. The parent arms of the two sessions agree
closely — 200/200 and 200/200 in protocol A, 179/200 against 174/200 strict in
protocol B — which is what makes the comparison usable at all.

The CSF container

Only the main-decoder routed-expert block scales are compressed: 73,728
planes, which is 48 layers × 512 experts × 3 projections. Each logical
weight_scale becomes two ordinary safetensors tensors.

MTP experts, attention, shared experts, the PLE n-gram table, every global scale
and every activation scale keep a plain weight_scale. config.json declares
weight_scale_encoding: "csf" on the 48 routed-expert recipes and on nothing
else.

Verification

Built and checked with remote/csf_codec.py, remote/build_csf_container.py
and remote/verify_csf_container.py in our pipeline repository. All checks are
exhaustive rather than sampled:

1. All 73,728 planes decode back to the source bytes exactly, read from the
   written shards rather than from build-time state.
2. All 227,865 other tensors are byte-identical to the uncompressed source
   build, dtype and shape unchanged, including the trained FP32 normalisation
   parameters.
3. The index maps every tensor, and weight_scale_encoding appears on
   routed-expert recipes only.

The builder additionally decodes every plane and compares it with the source
before writing the shard, so a mismatch aborts the build rather than shipping.

Serving

Needs a vLLM build with the NVFP4-CSF reader and the b12x SM120 kernels.
main is v2, so no revision pin is needed.

VLLM_PLE_TABLE_MEMORY=disk vllm serve lflare/Qwen3.8-Flash-Next-NVFP4-MXFP8-CSF-QAD-Abliterated \
  --quantization modelopt_mixed --load-format safetensors \
  --tensor-parallel-size 1 --pipeline-parallel-size 1 --dtype bfloat16 \
  --moe-backend b12x --linear-backend b12x \
  --mm-encoder-tp-mode data --mamba-cache-mode align \
  --enable-chunked-prefill --block-size 16

--quantization modelopt_mixed is auto-detected from config.json and may be
omitted. TP1, TP2 and TP4 all satisfy the scale-alignment requirement. Pipeline
parallelism must be 1. Expert parallelism, data parallelism and ubatching are
not supported on the CSF path.

A runtime must support CSF, and most do not. The scale planes are not plain
FP8, so a loader without a CSF path cannot read this repository at all. The
karmic-kraken-20261001 and 20260925 images have no CSF support;
karmic-kraken-20261016.2 does. Proof the CSF path actually ran rather than a
fallback: the autotuner issues quantization.csf_block_expansion requests at
startup.

Known dependency gap. The CSF scale loader imports b12x.loader._csf_blocks
and b12x.loader._csf_source, which are in no released package, so this path
needs a source build of local-inference-lab/flashinfer or one of their images.
If that is impractical, decode the planes back to plain weight_scale tensors
with our remote/csf_codec.py and drop weight_scale_encoding from
config.json — that yields a checkpoint which runs on any ModelOpt NVFP4 build,
at a cost of 3.58 GB.

These files cannot be loaded by Flare. Its runtime accepts only its own
prepared package format; see
lflare/Qwen3.8-Flash-Next-FLARE-Abliterated for a Flare package carrying
these weights.

Served and verified on GPU, 2026-10-19

This container has been loaded and served. An earlier revision of this card
said it had not, which was true when written.

That second row is the one v1 could not produce: v1 reaches 0 with thinking off
and still refuses with thinking on. The probe ran through the CSF container on a
different runtime from the one that produced the gate tables above, which makes
it an independent confirmation of the dual-regime result rather than a repeat.

The capability and held-out refusal numbers above still come from the
uncompressed build, served for ~1.5 hours across two concurrent arms. The CSF
encoding is lossless and verified byte-for-byte against that build, so those
numbers transfer; what the probe adds is proof that the CSF reader path
serves them, which byte-verification against our own source could never show.

Differences from Local Inference Lab's CSF-QAD container

Do not treat the two as interchangeable.

• The vision tower here is BF16. Their main revision additionally
  converted 110 vision linears to MXFP8 and NVFP4. This container inherits the
  unconverted vision weights of the 6909a5b parent, because our abliteration
  never touched vision and we did not reproduce their conversion.
• 40 shards, keeping the parent's shard names, where their main revision
  reshards to 21.
• No manifest.json, build-contract.json, receipts/ or verification.json.

Not verified

• No vision evaluation was run on any of our builds.
• The on-container refusal probe is n=20 per regime, which confirms the
  behaviour is live rather than measuring its rate. The rates come from the
  200-row and held-out-136 gates on the uncompressed build.

Safety

This model has had refusal behaviour deliberately reduced, and v2 reduces it in
both thinking regimes where v1 only did so with thinking off. It will answer
requests that the parent declines. Do not deploy it as a general-purpose
assistant without your own safeguards, and do not treat the refusal numbers
above as a safety property.

Report refusal numbers as a protocol and scorer pair, and compare them only
against the parent measured under the identical protocol and scorer.

Licence and credits

Qwen Community License 1.0, inherited from the parent. See LICENSE.

Condition 1 of that licence requires the notice to travel with every copy. If
you operate a Model-as-a-Service or AI-assistant business, condition 2 requires
a separate licence from Qwen before any commercial use.

• Qwen/Qwen3.8-Flash-Next — base model, by Qwen.
• local-inference-lab/Qwen3.8-Flash-Next-NVFP4 — the QAD NVFP4 and MXFP8
  quantisation this builds on, by Local Inference Lab, Inc.
• The NVFP4-CSF container format and the b12x kernels — by Local Inference
  Lab, Inc. Neither they nor Qwen endorse this model.