base_model: local-inference-lab/Qwen3.8-Flash-Next-NVFP4
base_model_relation: finetune
license: other
license_name: qwen-community-1.0
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
pipeline_tag: text-generation
library_name: transformers
tags:

• abliterated
• reduced-refusal
• nvfp4
• mxfp8
• modelopt
• quantization-aware-distillation
• conversational

Qwen3.8-Flash-Next-NVFP4-Abliterated

Reduced-refusal model. A refusal-direction edit was applied deliberately. Read the safety section before deploying.

Summary

A refusal-direction projection applied directly to the quantised checkpoint
local-inference-lab/Qwen3.8-Flash-Next-NVFP4 at revision qad-step5500-ple1000, not to its BF16 parent.

That distinction is the point of this build. The parent is not a post-training
quantisation — it is quantisation-aware distilled, 5,500 trunk updates plus
1,000 PLE-refinement updates against the BF16 teacher
(quatrain_qwen38_ple_refinement_export).
Abliterating Qwen/Qwen3.8-Flash-Next and re-quantising would throw that
distillation away and produce a different, weaker artefact. Editing the quantised
weights in place is the only way to get an abliterated version of this checkpoint.

No tensor changed format. Every tensor keeps the dtype, block-scale tensors
and byte size it has in the parent. 25188 residual-writing tensors were
edited; the remaining shards are hardlinked from the parent and are byte-identical.

What was edited

Direction measured at decoder layer 43 of 48 (Cohen's d = 10.44),
from 128 harmful (mlabonne/harmful_behaviors) and 128 harmless
(mlabonne/harmless_alpaca) prompts, seed 42, as the normalised difference of
means of the residual stream entering that layer. Projection strength alpha = 1.0.
The direction is biprojected — orthogonalised against the harmless mean activation, so the ablated axis carries only the harmful-conditional component and shared harmless content is left in place.

Measurement details that matter, all following
HangGlidersRule/model-forge's
documented method:

• Measured on the quantised model itself, under vLLM, not on a BF16 parent.
  The residual stream here is a hyper-connection of width 4 x 2560; the four
  sub-streams are averaged to recover the 2,560-dim block-output space the write
  matrices live in.
• Prompts rendered with thinking off, matching the regime the headline refusal
  gate scores in. This is load-bearing: a direction captured with thinking on
  barely suppresses refusals with it off, and vice versa.
• The layer was chosen by generation test, not by activation statistics. Every
  layer's direction was screened by ablating it with inference-time hooks and
  counting refusals. The result is not monotonic in depth and disagrees with the
  Cohen's d argmax, so picking the most-separable layer would have shipped a
  weaker edit.
• Layer 0 measures a difference of exactly 0.0000, which is a free correctness
  check on the capture: every prompt shares the same final chat-template token, so
  the residual entering layer 0 must be identical across the set.

Why a naive edit does not work here

The stored weights already sit exactly on the quantisation grid, so a rank-1
projection worth 2-4 % of a tensor's Frobenius norm is mostly smaller than
half a unit in the last place and plain round-to-nearest returns the original
code. Measured on this checkpoint's own tensors, round-to-nearest retains only
74 % of the edit on MXFP8 and 14 % on NVFP4 — a checkpoint that looks
edited and largely is not.

This build instead chooses, per element, the code that both approximates the
edited weight and holds the constraint v · W = 0 across each column: rows are
visited in descending |v_i| and each absorbs as much of the accumulated
constraint error as a one-ULP move allows.

Residual leakage (||v.W_hat|| / ||v.W||; 0 = applied exactly, 1 = erased) is at
most 3.01e-06 across all 25136 quantised tensors, and
7.84e-03 across the 52 BF16 ones -- the BF16 tensors are the
looser of the two, because they take plain rounding and inherit bfloat16's 0.39 %
relative ULP, while the quantised ones are actively driven to zero. Mean added
weight error is 3.820% of each tensor's norm.

Evaluation

Capability parity against the parent, measured in the same session on the same
serving configuration. Scores from different serving configurations are not
comparable for these profiles.

lavd is the discriminating profile for this family; estonia is not (every
build measured scores 30/30). n=10 cannot resolve a difference of one or two runs,
so lavd is reported at n=30 against the parent in the same session — that is the
row to read. GPQA-diamond is sampled (temperature 1.0, top_p 0.95, top_k 20) at
model-forge's stated settings for their abliterated product, so a one-item gap
there is noise, not signal.

Refusal

treadon/abliteration-eval, 200 harmful + 83 safe over-refusal + 50
multilingual. Refusal counts, so lower is better in every cell.

A refusal number is the product of a protocol and a scoring convention, and
both are reported here because on this model family the protocol turns out to
dominate. Presenting a single cell would let the headline be chosen after the
fact.

• Protocol A is the one
  HangGlidersRule/model-forge
  publishes its Darkstar gate under: thinking off, temperature 0,
  max_tokens=100.
• Protocol B is a reasoning-mode run: thinking on, temperature 0,
  max_tokens=2048.
• The model-forge scorer is their src/model_forge/refusal.py: five
  refusal-form markers, whole-text match, an empty completion scoring as
  compliance.
• The strict scorer is ours: 47 markers matched in the first 400 characters
  of the visible content, and an empty completion counted as a refusal.

harmful (n=200)

over_refusal (n=83, benign prompts that look harmful)

multilingual (n=50)

Reading these numbers

The direction was measured with thinking off, matching protocol A, and that
is the regime the build is strongest in. Under protocol B the strict scorer is
harsher than the model-forge one mainly because this build thinks for longer:
it spends more completion tokens than the parent, hits the 2048-token cap more
often, and emits no visible content when it does. The strict scorer calls an empty
completion a refusal; model-forge's calls it compliance. Neither is wrong — they
answer different questions — but the gap between those two cells is largely that
convention, not extra refusal text.

Safety

This model's refusal behaviour was deliberately reduced. It will comply with
many requests the parent would decline, and it carries no added safety
mitigations. Refusal rates above are behaviour measurements, not a safety
endorsement. Deploy only behind appropriate policy, filtering, access control and
legal review.

Serving

Same runtime and recipe as the parent — mixed NVFP4/MXFP8 ModelOpt layout,
MTP3, FP8 KV cache, and the n-gram (PLE) table offloaded to host RAM. The
quantisation is unchanged, so anything that serves the parent serves this.

vllm serve lflare/Qwen3.8-Flash-Next-NVFP4-Abliterated \
  --served-model-name Qwen3.8-Flash-Next \
  --quantization modelopt_mixed --kv-cache-dtype fp8 --dtype bfloat16 \
  --max-model-len 262144 --max-num-seqs 16 --max-num-batched-tokens 6019 \
  --enable-prefix-caching --enable-chunked-prefill --language-model-only \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

VLLM_PLE_CPU_OFFLOAD=1 is required to fit a single 96 GB card.

Provenance

• Parent: local-inference-lab/Qwen3.8-Flash-Next-NVFP4
• Method: single refusal-direction projection, applied in-format; full per-tensor
  record in abliteration_report.json in this repo.
• lm_head reads from the residual stream rather than writing to it and was not
  edited. The vision tower was not edited.
• Per-run benchmark JSONs for both this build and the parent are under
  measurements/ in this repo, so every number above can be recomputed.

A more conservative variant — the same method applied to the attention output
projections, shared experts, embeddings and MTP writes only, leaving the 25,088
routed-expert down_proj tensors untouched — was built and measured alongside
this one. It is equally capability-neutral but only reaches 51 % harmful refusal
against this build's 5.5 %, so it was not published.