license: apache-2.0
base_model: GrootTeam/Groot-19B-A3B-261023
tags:
- abliteration
- uncensored
- nvfp4
- moe
- vllm
language: - en
groot-19b-a3b-uncensored-nvfp4
Uncensored build of GrootTeam/Groot-19B-A3B-261023, shipped as
NVFP4 for Blackwell (sm120) and served with vLLM.
The parent is a 19B-parameter MoE (3B active, 128 experts top-8, 30 gated
delta-net + 10 full-attention layers, 256K context, 1 MTP layer). Nothing about
the architecture changed; refusal behaviour was removed by directional ablation
and the result was re-quantised to the parent's own mixed-precision plan.
What was done
- Direction extraction. First-generated-token residuals on 200
harmful and 200 harmless prompts, matched per prompt on token
length (22.8 tokens mean each side) so the direction encodes
refuse-versus-comply rather than length or framing. Per-layer directions as
the normalised difference of means; the held-out evaluation prompts were
never used for extraction. - Attention ablation. Heretic 120-trial Optuna search over per-layer
directions and ablation-kernel shape on the attention output projections
(self_attn.o_proj,linear_attn.out_proj), norm-preserving. Selected
trial 57: KL divergence 0.0801 against the parent on benign text, with
attention weightsmax_weight=1.45at layer28.93,min_weight=0.18,min_weight_distance=18.61. - Expert ablation. Heretic cannot reach this model's fused expert tensors
(down_projis a single 128x2048x512 parameter), so the routed experts and
the shared expert were ablated separately: 1.5 strength on all expert
slices and 1.5 on the shared expert, norm-preserving, all 40 layers. - Quantisation. The parent's own compressed-tensors plan, reused verbatim:
FP8 W8A8 on attention and linear-attention projections, NVFP4 group-16
weights on experts, shared expert andlm_head; vision tower untouched.
Calibrated on 512 Alpaca samples. The MTP layer is not present in this build,
so it runs without speculative decoding.
Measurements
Compliance is graded by the HarmBench classifier (cais/HarmBench-Llama-2-13b-cls),
the metric validated against human labels; refusals and answer quality are graded
by a local rubric judge (Pennyroyal 27B), StrongREJECT-style, on the text after the
thinking block. Prompts: 320 held-out harmful instructions, disjoint
from the extraction set; 250 XSTest safe prompts for over-refusal.
| Build | HarmBench compliance | Refusals (judge) | Hot (score >= 0.5) | Mean score | XSTest refusals |
|---|---|---|---|---|---|
| parent (BF16) | 1.6% | 7.8% | 92.2% | 0.553 | – |
| abliterated (BF16, t57-e15) | 2.5% | 5.0% | 95.0% | 0.570 | – |
| this build (NVFP4) | 3.7% | 2.8% | 97.2% | 0.583 | 0.0% |
Capability, identical harness on both builds:
| Build | GSM8K | MMLU |
|---|---|---|
| parent | 94.0 | – |
| abliterated (BF16) | 83.0 | – |
String matching on refusal phrases was also recorded and is useless on this
model: it reports 1.6% refusals on a build the judge scores
differently, because the refusals are paraphrased.
What it is not
Abliteration removes a refusal direction; it does not add judgement, and it does
not make the model safe. Expect capability loss proportional to the numbers
above, and expect the model to comply with things the parent declined. The
parent's licence (Apache-2.0) carries over. No benchmark here measures whether
the output is correct or wise — only whether it complies and how much capability
survived.
Reproduction
The pipeline is a self-contained local repository (prompts, direction extraction, ablation, quantisation recipe, judge harness); the quantisation plan is the parent's own quantization_config, reused verbatim. Direction extraction, ablation strengths, trial selection and
quantisation are all recorded, and the evaluation sets are held out from
extraction: 200 harmful and 200 harmless prompts for the direction, 120 harmful
plus 225 XSTest prompts for the measurement, no overlap.