license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/license/1-1/
base_model:

• nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16
  base_model_relation: finetune
  quantized_by: AtomicChat
  pipeline_tag: text-generation
  library_name: gguf
  tags:
• atomic-chat
• atomic-dynamic
• nvidia
• nemotron
• gemma4
• gguf
• imatrix
• llama.cpp
• abliteration
• refusal-direction

How to Run NVIDIA Nemotron 3.5 Bolt E4B Without Refusals Locally

NVIDIA's refusal behaviour projected out of the released weights, baked into Atomic Dynamic GGUF builds and shipped as a 3.5 MB LoRA, measured against the original.

• On held-out JailbreakBench prompts refusals fall from 93.8% to 1.2%, on Russian red-teaming prompts from 76% to 0%, on the safe XSTest prompts from 9.2% to 0.4%, and with thinking on from 29 of 30 to none. MMLU: 68.50% before, 67.85% after (2000 questions, McNemar p = 0.21). Tool calls and long context are unaffected.
• One release-gate is not met, and we say so: on harmless prompts the first token moves further from the original than our 0.10 limit (KL 0.141). On this model even a random direction edited the same way moves neutral text noticeably (limitations).
• Gemma 4 normalises every write into the residual stream with its own weights before adding it, so the edit is computed through those norms: each matrix stops writing the refusal direction as it lands in the stream, not before the norm.
• Built on the same weights as our Nemotron 3.5 Bolt E4B quants; the LoRA loads on top of any of those files.

The builds

KLD and top-1 are against the original, unmodified BF16 model on our held-out
neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization
together. The edit alone, on the BF16 weights, is 0.159; the plain AD-Q6_K of the
original model is 0.026 and AD-Q4_K_M 0.091 on the same text, so at AD-Q6_K almost
all of the distance is the edit itself.

The baked files themselves, thinking off, on the held-out prompts: each refused 1 of
the 81 English, none of the 100 Russian and 1 of the 250 XSTest prompts, with no empty
or degenerate replies. Thinking on was checked on the edit (step 5), not on the baked
files.

The LoRA. NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-LoRA.gguf is the same edit as a
rank-1 adapter (126 matrices, 3.5 MB). It is built for scale 1.25, so load it with
--lora-scaled, not --lora (which applies 1.0):

llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-AD-Q4_K_M.gguf \
  --lora-scaled NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-LoRA.gguf 1.25 \
  -ngl 99 -c 32768 --jinja

Every number in this card for the edit itself was measured with the adapter on
the BF16 weights; on a quantized base it was not measured separately.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-AD-Q6_K-00001-of-00001.gguf \
  -ngl 99 -c 32768 --jinja --temp 1.0 --top-p 0.95

The chat template is inside every file; thinking is off unless you pass
--chat-template-kwargs '{"enable_thinking": true}'. Upstream llama.cpp with
gemma4 support runs them; no patch is needed. For images and audio, use the
projector from the original quants
(the edit does not touch it); we did not test them on these files.

How we did it

Bolt E4B is NVIDIA's post-train of Gemma-4-E4B-it: 42 blocks, hidden size 2560,
per-layer embeddings, and every block writing into the residual stream three
times. The usual abliteration recipes assume a plain transformer and miss two of
those writes and the norms around all three.

1. Refusal is a direction

A chat model decides to refuse before it writes the first word, and that decision
shows up as one direction in its residual stream (Arditi et al., 2024). Stop the
weights from writing along that direction and the model stops refusing.

2. Finding it

• Prompts. 416 harmful and 416 harmless English prompts (harmful_behaviors,
  harmless_alpaca), 128 and 128 Russian ones (Aya red-teaming, Russian Alpaca),
  each in the model's own chat template, thinking off. Validation and test prompts
  are separate sets and never reach this step.
• Direction. At each depth: mean over harmful prompts minus mean over harmless
  ones, English and Russian pooled, with the component along the harmless mean
  removed (the naive difference makes models refuse harmless look-alikes).
• Depth. How well the direction separates the two sets and how close the
  English and Russian directions are:

We took block 32, three quarters of the depth, as on our other releases: one
direction for both languages, separation near its peak. Block 21 separates as
well but its edit left 68% of the English validation prompts refused.

3. Where to cut, through the norms

Each Gemma 4 block writes three times: attention through attn_output, the FFN
through ffn_down, and the per-layer path through its projection proj (each
token's per-layer embedding, gated, 256 wide, back to 2560). Every one of the three
passes its own RMSNorm with a learned weight g before it is added to the stream.
Removing r from a matrix's output is not enough there: the norm multiplies each
coordinate by its weight, and the direction comes back. So for each matrix we remove
r ⊙ g (normalised) instead: then g ⊙ y has no component along r, and the
norm's scalar divisor cannot add one. 126 matrices (3 × 42); the embeddings and the
head are left alone.

Measured with the adapter at scale 1.0 on a text the edit never saw: what each
family still writes along r, as a share of its output, is 0.00023 (attention),
0.00021 (FFN) and 0.00019 (per-layer path), against 0.001 as our limit.

4. The edit

For every one of the 126 matrices:

W' = W - s · r' r'ᵀ W        r' = (r ⊙ g) / |r ⊙ g|

screened as a rank-1 adapter on the BF16 weights:

1.25 is the smallest scale that clears both languages; every larger one moves the
model further and costs MMLU. The random direction, edited through the same norms at
the same scale, leaves the refusals as they were, so the effect comes from the
direction.

5. Checks before release

The numbers in this card come from test prompts that played no part in any choice:
81 JailbreakBench behaviours, 100 Russian red-teaming prompts and the 250 safe XSTest
prompts. The thresholds, and where this edit stands:

6. Baking and quantizing

The edit is applied to the BF16 weights in f32 and rounded to BF16 once. After
rounding, every one of the 126 matrices keeps a quarter of its write along r' (the
ideal 0.25) within 0.0011. The edit moved the importance matrix (cosine below 0.999 on
300 chunks), so it was collected again on the baked weights, over our nemotron-bolt-e4b corpus, and the baked BF16 was quantized with the same Atomic Dynamic
layouts as those quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an
adapter), on the test prompts of step 5.

Refusal is counted by the opening of the reply, so it is indicative, not a judge.

Limitations

• The harmless-prompt gate is not met. The edit moves the first token on harmless
  prompts by KL 0.141, over our 0.10 limit, and neutral text by KLD 0.159. This model is
  unusually sensitive: a random direction edited the same way already moves neutral text
  by 0.125 and costs 1.2 MMLU points, more than the refusal direction does. A smaller
  scale did not clear the refusals in our screen.
• The refusal count reads the opening of each reply; a soft refusal phrased as an
  answer would be missed.
• MMLU and the thinking check use fixed question sets; they show a large change, not a
  small one.
• Image and audio input were not tested.

Responsible use

This model has its learned refusal behaviour removed, so it will answer requests the
original declines. Nothing about it makes the output safe, correct or lawful, and any
deployment needs its own access controls and policy enforcement.

License and credits

Use of the model is governed by NVIDIA's
OpenMDW License Agreement, version 1.1.
NVIDIA post-trained it from Google's Gemma 4 (Gemma-4-E4B-it), whose use is governed by the
Apache License, Version 2.0. Base model
Nemotron 3.5 Bolt E4B by NVIDIA. Direction, edit,
quantization and evaluation by AtomicChat. The method follows Arditi et al.,
Refusal in Language Models Is Mediated by a Single Direction (2024).