base_model:

• nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUF
  base_model_relation: finetune
  license: other
  license_name: openmdw-1.1
  license_link: https://openmdw.ai/license/1-1/
  pipeline_tag: image-text-to-text
  library_name: gguf
  language:
• en
• es
• fr
• de
• it
• ja
  tags:
• gguf
• llama.cpp
• nemotron-3.5
• latent-moe
• moe
• mixture-of-experts
• mamba
• hybrid
• multimodal
• abliterated
• uncensored

NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Abliterated-24GB-GGUF

An uncensored edit of NVIDIA Nemotron-3.5-Bolt-74B-A9B, a hybrid Mamba/MoE model, produced by
editing the shipped 24GB 2-bit GGUF directly — no BF16 requantize at any point.

That matters for this model specifically. Bolt is a LatentMoE: every routed expert sums through one
dense ffn_latent_up per layer, and the experts themselves are ~17GB of IQ2_XXS weights produced by
NVIDIA's quantization-aware distillation. A conventional abliterate-then-requantize pipeline would
throw that QAD work away. Instead the projection is applied in place and codes are re-picked under
the original block scales, so at λ=0 the output file is byte-identical to the input by
construction — which is the self-test the build gates on (verified: all edited tensors
byte-identical at λ=0). The published file is the same length as NVIDIA's, 22,018,326,528 bytes.

Files

Bolt is multimodal, so the projector ships alongside. The edit touched only text-model tensors, so
the projector is byte-for-byte NVIDIA's file; download both or vision silently does nothing.

llama-server \
  -m NVIDIA-Nemotron-3.5-Bolt-74B-A9B-Abliterated-24GB.gguf \
  --mmproj mmproj-NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf \
  -ngl 99 -c 32768 --jinja

Any llama.cpp build that loads NVIDIA's own 24GB GGUF loads this one: same architecture, same
tensor layout, same file length.

Give it at least 8192 tokens of budget — this is the one setting that matters. At 8192 it
answered 154 of 154 held-out harmful prompts without hitting the cap once; at 4096 it capped 11 of
150. For long benign answers it needs no more room than stock does, and both cap about 20% of those
even at 8192, so go higher still for essay-length work.

The edit

Read-side projection W ← W − λ(Wr)rᵀ over the input-projection surface, blocks 15–75:

λ=1.0, with λ=2.0 on ssm_in. 100 tensors edited in total.

Realised removal is measured on the decoded bytes, not assumed from the arithmetic, and the two
differ a lot here. ffn_gate_inp is stored F32, so its edit is exact and perfectly linear in λ. The
Q8_0 tensors reach 0.92–0.94 rather than 1.0 because Q8_0 already scales so the largest code is
127, and subtracting the projection pushes a small fraction (~0.5%) of values out of range. ssm_in
is NVFP4, whose coarse grid rounds most of the edit away.

That last one is why λ=2.0 appears only on ssm_in. Over-driving λ cannot defeat a quantization
grid — since the target is (1−λ)·rᵀW, intended removal peaks at λ=1 and over-driving walks off
the far side, with large λ eventually amplifying refusal rather than removing it. NVFP4's
under-realisation partially cancels that overshoot, which makes ~2.0 the empirical optimum at ~0.5
realised. It is a ceiling, not a tuning knob: no λ gets this tensor to 0.95.

Direction from a mean-difference control vector (the Arditi et al. 2024 method), last-token
estimator, taken at layer 45.

Evaluation

Read the answer rate before the refusal rate

A refusal rate on its own is close to meaningless, and this is not a pedantic point — it bit this
very build. An earlier candidate scored 0.67% refusal and looked like the obvious winner. Its
score shape showed it answered only 105 of 150 prompts: it thinks until the token budget runs
out, and an empty answer is scored 0 by the judge in exactly the same way a refusal is. Most of that
beautiful 0.67% was the model saying nothing at all.

So a low refusal number is only interpretable next to how often the model actually answered and
how often it hit the cap. Both columns are below for every number reported here. If you are
comparing this model against another abliteration, ask for those two columns; if they are not
published, the refusal percentage is not a measurement you can use.

That is not a hypothetical worry, and the cost is measurable. The same candidate was also run on
HumanEval, where it scores 85.4% against stock's 93.9% — paired McNemar p = 0.0005, a real
capability loss — while the model released here sits at 91.5% with p = 0.29. The mechanism is
visibly identical in both places: conditional on producing an answer at all, that candidate passes
97.9% of problems, essentially the same as this model's 98.0% and close to stock's 99.4%. It is not
worse at writing code. It caps 21 of 164 problems where stock caps 9. The refusal benchmark and
the coding benchmark were measuring the same stall, and only the answer-rate column makes that
visible. Picking the winner on refusal percentage alone would have shipped the weaker model.

Both candidates were then re-run on the held-out half of StrongREJECT at an 8192 budget, which
is the cleanest form of the comparison — same prompts, nothing selected on, enough budget for either
to finish:

Read the first row alone and the other candidate is a flawless abliteration and this model is a
failure. Read any other row and it is the reverse. It refuses nothing because it finishes nothing:
it thinks for 12,275 characters at the median, caps 39 times even with 8192 tokens, and leaves 38
prompts unanswered — each of which the refusal classifier scores as "did not refuse". A 0% refusal
rate is what a broken model looks like, which is exactly why the answered and cap columns belong
beside every refusal number anyone publishes, including the ones above.

All refusal numbers come from an anchor-gated frozen judge (Qwen3.8-27B-OBLITERATED-Q8_0, thinking
off, anchor 0.6875 ± 0.0051; every pass in these tables gated at 0.6863). Higher StrongREJECT mean
score = more complete compliance.

Selection set — the prompts this build was chosen against

Held out — never used to select anything

Separate prompts, generated and judged after the build was frozen.

Refusal 154/154 → 18/154 is z = 15.61, p = 6.7e-55. The 11.7% held-out refusal sits 1.0pp from the
10.7% it was selected on, inside the 6.0pp resolution limit, and the mean score is marginally
higher on the held-out half — so this did not fit the selection set.

Stall control — long benign answers (n=150, held out)

Abliteration can leave a model unable to stop. That shows up as nothing at all on a refusal
benchmark and ruins the model in use, so it is measured directly: 150 benign prompts that each
demand a long answer, same 8192 budget.

No stall. This model thinks 4.1% longer than stock and caps slightly less often. The third
column is why that null is worth printing rather than just an underpowered shrug: the same test, on
the same prompts, detects a +67.9% thinking increase in the other candidate. A test that can
register a two-thirds increase and reports 4% is reporting something.

Two honest limits on this table. The cap-rate row resolves almost nothing — at n=150 against a
20% baseline, a two-proportion difference needs roughly 13pp to be detectable, so none of these
cap-rate gaps mean anything; the median-think row is a continuous measure and is far better resolved,
which is why the argument rests on it. And note that stock itself caps 20% of these prompts at
8192: this prompt class wants more budget than 8192 for either model, which is a property of the
task rather than of the edit.

Capability — HumanEval, paired (n=164, held out)

Both arms ran the same 164 problems, so these are paired and the right test is McNemar on the
problems where the two disagree — not a two-sample proportion test, which would discard the pairing.

Paired outcome: 148 both pass, 8 both fail, 6 regressed, 2 gained. Exact McNemar on the 8
discordant pairs gives p = 0.29 — no detectable capability difference.

Be careful how much that p-value is asked to carry. With only 8 discordant pairs, significance
would have required a split of at least 1–7; a 6–2 split is simply what a coin does fairly often. So
this rules out a lopsided regression and nothing finer. What is more informative than the p-value
is the decomposition: 11 of this model's 14 failures are budget caps, not wrong answers (stock
caps 9 of its own), and on problems where it produced an answer at all it passed 150/153. The
headline 2.4pp gap is mostly the 4096-token ceiling, which constrains both arms.

The independent benchmark is the important row, and it is also the softer one. Every other
harm number here is StrongREJECT; the held-out tail is the same dataset, so it tests generalisation
to new prompts, not to a new benchmark. forbiddenquestions is a dataset nothing in this work
selected on, tuned against, or had ever generated before this run — and on it, stock refuses only
65.7%, complying with about a third outright. So the 61.8pp drop there is measured from a much
weaker baseline than the 88.3pp drop on StrongREJECT, and the two are not interchangeable evidence.
What it does establish is that the effect is not an artifact of one benchmark.

Resolution limits were fixed before any of these were run (80% power, two-sided) — but they were
all computed assuming a ~99.3% stock baseline, which forbiddenquestions does not have. Both the
pre-registered and the corrected figure are given, because the pre-registration is the honest record
and the correction is the number that applies:

The forbiddenquestions floor was pre-registered 3.3× too optimistic. The effect is still 6× the
corrected floor, so nothing here changes — but the 3.1pp figure should not be quoted. Note also that
the over-refusal improvement (8.0pp against a 7.1pp floor) only just clears its own limit; treat it
as "does not over-refuse more than stock" rather than as a reliable improvement.

Honest caveats

It thinks longer than stock on harmful prompts — but that is not a general slowdown, and the
3.2× figure is misleading on its own. On the harmful set, median reasoning is 4,208 characters
against stock's 1,327. Stock's number is small because stock refuses, and a refusal is cheap to
write; setting that against a model that actually engages with the question compares two different
tasks. The controlled comparison is the stall test above, where the two sit within 4% of each other.
And the stall that looked real was a budget artifact. Those 11 capped prompts were run at a 4096
token budget. Re-run on the held-out half at 8192, this model caps 0 of 154 and answers
every one — so give it 8192 and the defect disappears. That is not true of every candidate: the
0%-refusal one still caps 39 of 154 at 8192 and answers only 116, which is a real stall rather
than a budget ceiling.

Capability: no detectable loss — on tests that could only have caught a large one. HumanEval's
paired test gives p=0.29, but with 8 discordant pairs it would have needed a 1–7 split to register,
so it does not establish that nothing changed. MATH-30 (21/23 against stock's 23–24) cannot resolve
anything under ~33pp. Together they rule out collapse and a lopsided regression; a few points of
real degradation would pass through either test unseen. Treat capability as "not visibly harmed",
and if you need a tighter bound, run your own benchmark at a budget above 4096.

The 0% over-refusal is on 100 prompts. At a 6% baseline that test can barely resolve anything;
only an extreme is interpretable. The held-out n=150 arm is the one to quote.

Single judge. Every refusal number comes from one frozen judge model. It is anchor-gated for
drift, which is not the same as being unbiased.

Vision is untested beyond loading. The projector is unmodified and the model loads and answers
with it, but no image benchmark was run. Nothing here measures multimodal quality.

What is and is not new here

Cited, not claimed:

• Arditi et al. 2024, refusal as a single mediating direction — the method this uses.
• Locating and Steering Refusal Beyond Attention already
  showed refusal concentrates in the Mamba path of a hybrid (Zamba2) rather than attention, and
  already made the "decodable but not controlling" argument. This work replicates the first on a
  different hybrid; neither is claimed as new.
• Weight-space abliteration of a hybrid Mamba/MoE model is not new either — and the earlier work
  is my own.
  Nemotron-3.5-Lightning-30B-A3B-Uncensored-ShimQuant-GGUF
  already projected the refusal direction out of a nemotron_h hybrid's attention, Mamba and
  shared paths. It is fair to say the published literature still contains no weight-space hybrid
  abliteration — the Zamba2 work above is activation-space only, and the 16-model cross-architecture
  survey (2512.13655) marks Mamba-containing models
  architecturally incompatible with every tool but one, covers no hybrid and does not evaluate MoE.
  But a shipped model is prior art for the next one, including your own, so this card does not claim
  the hybrid edit as a first.

What is actually new here, measured against both papers and that earlier build:

• Doing it inside a shipped 2-bit quant. Lightning was edited in BF16 and then quantized; this
  edits NVIDIA's released 24GB file in place under its original block scales, so the QAD experts are
  never touched and λ=0 is byte-identical by construction.
• A maximum-reach negative control. Editing 106 tensors spanning every class except the Mamba
  path, across both the read and write surfaces, leaves refusal at 90.0% with the model perfectly
  healthy (150/150 answered, normal reasoning length, 0% over-refusal). That turns "Mamba has the
  larger effect" into "everything else, at maximum reach, does nothing."
• The MoE router dissociation. The F32 router takes a mathematically exact edit (removal 1.000),
  demonstrably changes which experts fire (only 52.6% of pick sets identical), and refusal does not
  move at all — four separate judged nulls. Expert selection is not where refusal lives.
• Direction interacts with surface. The write side wants an all-token direction from layer 23;
  the read side wants a last-token direction from layer 45. Each is near-null on the other's
  surface. A single "best direction" per model is the wrong abstraction.

A result retracted, recorded here because the retraction is the useful part. Harmfulness was
measured as linearly readable at AUC ~1.0 in a late block band, and that was nearly published.
Control blocks chosen because they had not ranked separated just as well (shortlist vs control
p=0.475), and isotropic random directions reached |AUC−0.5| of 0.47 against the real direction's
0.50. The prompt pair (AdvBench vs Alpaca) is separable along almost any projection. A cross-corpus
replication had looked excellent — an artifact of the measurement replicates perfectly, because the
measurement is what is broken. No readability or localisation claim is made.

Intended use and limits

This model will answer harmful requests. It is published for safety research, red-teaming, and
evaluation work where a refusal-free baseline is the point. Refusal removal is not a capability
gain; it removes a safety behaviour and nothing else. Anyone deploying this to users owns the
consequences and needs their own safeguards.

Related build:
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-Uncensored-ShimQuant-GGUF
— the same family, a smaller hybrid, edited in BF16 and then quantized rather than in-quant.

Licensed under OpenMDW 1.1, inherited from the parent model.