license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/license/1-1/
base_model:

• nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
  base_model_relation: finetune
  quantized_by: AtomicChat
  pipeline_tag: image-text-to-text
  library_name: gguf
  tags:
• atomic-chat
• atomic-dynamic
• nvidia
• nemotron
• nemotron_h_moe
• moe
• gguf
• imatrix
• llama.cpp
• abliteration
• refusal-direction

How to Run NVIDIA Nemotron 3.5 Super VL 120B-A12B Without Refusals Locally

NVIDIA's refusal behaviour projected out of the released weights, baked into Atomic Dynamic GGUF builds and measured against the original.

• On held-out JailbreakBench prompts refusals fall from 95.1% to 0%, on Russian red-teaming prompts from 76% to 2%, and with thinking on from 30 of 30 to 2 of 30. MMLU stays where it was: 85.05% before, 85.10% after (2000 questions, McNemar p = 1.0). Tool calls and long context are unaffected; no empty or looping replies.
• The refusal is written into the residual stream by every kind of block: the edit covers the Mamba-2 outputs, the attention outputs, the shared experts and the latent projection through which all 512 routed experts write.
• Same Atomic Dynamic layouts and importance matrix as our Nemotron 3.5 Super quants; each file is the size of the original build with the same name.

The builds

KLD and top-1 are against the original, unmodified BF16 model on our held-out
neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization
together. For scale, our plain builds of the original model with the same layouts
are 0.0082, 0.0352 and 0.0722 (on 95 chunks): the edit itself adds about 0.004.

The baked files themselves, on the held-out prompts: AD-Q4_K_S refused 0 of 81
English, 3 of 100 Russian and 1 of 250 XSTest prompts; AD-IQ3_XXS 0, 2 and 1;
AD-Q2_0_L 0, 5 and 0.

The vision projector (mmproj-*.gguf) is the original model's, unchanged; image
input was not tested on these files.

Running it

hf download AtomicChat/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-GGUF \
  --include "NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/*" --local-dir .
llama-server -m NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-Uncensored-AD-IQ3_XXS-00001-of-00043.gguf \
  -ngl 99 -c 32768 --jinja

The chat template is inside every file. Upstream llama.cpp with nemotron_h_moe
support runs them; no patch is needed.

How it was made

1. Direction. Difference of means of the residual stream at the last prompt
   token, thinking off, chat template applied: 416 harmful against 416 harmless
   English prompts and the same in Russian, on the released weights. The
   component along the harmless mean is removed. One direction, measured at the
   input of block 66 of 88.
2. Edit. W' = W - r rᵀW on the 128 matrices that write into the residual
   stream: the output projection of all 40 Mamba-2 blocks and of the 8 attention
   blocks, the down projection of the 40 shared experts, and the 40 latent up
   projections through which every routed expert writes. The router, the token
   embeddings and the multi-token-prediction head are untouched.
3. Choice. The recipe was set from a rehearsal on a smaller model of the same
   family and confirmed here against an alternative (block 50) and a random
   direction of the same size as the control, on validation prompts and then on
   every check below.
4. Bake. The same edit applied to the BF16 weights in f32 and rounded once,
   then quantized with the importance matrix and the layouts of our
   original quants.

What changed, and what did not

Original against this edit on the released weights (BF16, the edit applied as an
adapter). The refusal sets are held-out prompts never used to pick anything.

Validation prompts (104 harmful, 104 harmless per language), replies of up to 64
tokens, the check of the recipe on the released weights: 1.9% of English and 3.0%
of Russian harmful prompts refused.

Refusal is counted by the opening of the reply, so it is indicative, not a
judge.

Limitations

• With thinking on, 2 of 30 prompts were still refused; our gate allows fewer.
  We publish because every other check passed and the difference from zero is
  inside the noise of 30 prompts, but the thinking mode is not fully cleared.
• The refusal count reads the opening of each reply; a soft refusal phrased as
  an answer would be missed.
• MMLU and the thinking check use fixed question sets; they show a large change,
  not a small one.
• We publish the baked files only. The same edit as a LoRA adapter needs a
  llama.cpp patch on this architecture (upstream applies no LoRA to the latent
  projections), so it is not offered here.

Responsible use

This model has its learned refusal behaviour removed, so it will answer
requests the original declines. Nothing about it makes the output safe, correct
or lawful, and any deployment needs its own access controls and policy
enforcement.

License and credits

OpenMDW-1.1, as the original. Base model Nemotron 3.5 Super VL 120B-A12B by
NVIDIA. Direction, edit, quantization and
evaluation by AtomicChat. The method follows Arditi et al., Refusal in
Language Models Is Mediated by a Single Direction (2024).