license: apache-2.0
base_model: sensenova/SenseNova-Loop-9B
base_model_revision: 329a65f701ba8c48b73c30d0cbaa6f0449733673
library_name: transformers
pipeline_tag: text-generation
tags: [nvfp4, quantized, abliterated, sense-nova-loop]
SenseNova-Loop-9B abliterated (NVFP4)
The QAD build with refusal behaviour removed by training, not by a weight projection. Every linear
residual-direction method failed on this checkpoint (diff-of-means, refusal-margin regression, logit-lens,
multi-category, baked steering: 40+ layer/strength points either left 16 of 16 validation refusals or
destroyed the model at KL 2-14). The working recipe is an unlikelihood objective: LoRA r64 on the quantized
linears, minimising logsumexp(logits[refusal-opener ids]) at the last prompt position of the training
harmful prompts, early-stopped on validation (0 of 16 refused at step 25), folded into the masters and
requantized to NVFP4.
| Build | Harmful refused (of 99) | Benign refused (of 100) | MMLU | WikiText NLL |
|---|---|---|---|---|
| NVFP4 base (QAD) | 95 | 60 | 43.0% | 2.7560 |
| abliterated | 3 | 7 | 58.0% | 2.7619 |
Sealed JailbreakBench+AdvBench prompts, greedy 96 tokens, thinking off; MMLU 100-row slice; NLL over 24
WikiText windows of 1,024 tokens; bf16 reference NLL 2.8206. Harmful refusals fell
from 95 to 3 and benign refusals from 60 to 7, while MMLU and NLL improved rather than degraded.
Weights: NVFP4 W4A16, group 16, every language-model attention/MLP linear; embeddings, LM head, norms, mHC
routers (fp32) and the vision tower stay bf16. load_nvfp4.py loads the packed build, or decode it to bf16
(decoding is exact, so the forward values match the numbers above).
Limits
The refusal-removal SFT saw the program's training harmful prompts (810), not the sealed set. It is a
behavioural edit, not a capability change: the model is not safety-aligned and should be used accordingly.