license: other
license_name: openmdw-1.1
license_link: LICENSE
base_model:
- bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B
- trohrbaugh/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16-heretic
base_model_relation: merge
library_name: transformers
pipeline_tag: text-generation
language: - en
tags: - nemotron_h
- nemotron-3.5
- thinkingcap
- abliterated
- uncensored
- heretic
- moe
- mamba
- mtp
- bf16
ThinkingCap-Nemotron-3.5-Lightning-30B-A3B-abliterated
An uncensored variant of bottlecapai/ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B, a Nemotron 3.5 Lightning 30B-A3B fine-tune with shorter thinking traces. The refusal behaviour was removed by transplanting a published abliteration of the base model onto ThinkingCap; the MTP head for speculative decoding is carried over unmodified. Full bf16 weights.
This is the source checkpoint. The quantized builds made from it:
| Build | Size | KL vs. this model | Notes |
|---|---|---|---|
| This repo, bf16 | 65.8 GB | — | source; quantize from here |
…-abliterated-GGUF |
20.3–35.0 GB | — | Q8_0 and two per-tensor layouts, llama.cpp |
FP8-DYNAMIC, NVFP4A16 and W4A16 builds for vLLM are in testing and will be listed here.
Usage
vLLM
vllm serve IstroSec/ThinkingCap-Nemotron-3.5-Lightning-30B-A3B-abliterated --async-scheduling \
--reasoning-parser nemotron_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder \
--speculative-config '{"method":"mtp","num_speculative_tokens":5}'
Sampling (NVIDIA's recommendation for the base model, which ThinkingCap uses unchanged): temperature 1.0, top_p 0.95. Thinking is on by default; turn it off per request with chat_template_kwargs: {"enable_thinking": false}.
MTP. The head is present and unmodified. As a reference point: on the original ThinkingCap weights, bottlecapai measured a mean acceptance length of 4.01 with 5 drafted tokens (vLLM 0.27.1, B200). A value near 1 means the head is missing or mis-loaded.
Memory. On GPUs with less memory, vLLM can stop at startup with max_num_seqs (1024) exceeds available Mamba cache blocks; lower --max-num-seqs.
Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("IstroSec/ThinkingCap-Nemotron-3.5-Lightning-30B-A3B-abliterated", dtype="bfloat16", device_map="cuda")
tok = AutoTokenizer.from_pretrained("IstroSec/ThinkingCap-Nemotron-3.5-Lightning-30B-A3B-abliterated")
msgs = [{"role": "user", "content": "…"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to("cuda")
out = m.generate(ids, max_new_tokens=4096, do_sample=True, temperature=1.0, top_p=0.95)
llama.cpp
Use the GGUF builds; they were converted from this checkpoint.
Evaluation
Heretic's evaluation protocol, 100 harmful prompts from mlabonne/harmful_behaviors and 100 harmless from mlabonne/harmless_alpaca, against the original ThinkingCap, with thinking off (enable_thinking=False):
| Refusals | KL divergence vs. original | |
|---|---|---|
| ThinkingCap-Nemotron-3.5-Lightning-30B-A3B (original) | 99 / 100 | — |
| This model | 21 / 100 | 0.0458 |
Heretic itself cannot load this architecture, so the numbers come from a standalone port of its evaluator (Heretic 1.2.0's prompts, chat-template call, greedy 100-token responses, refusal markers and first-token KL, line for line). KL is measured on first-token distributions over the harmless prompts.
Two runs of the same weights gave 21 and 23: between them 10 of the 100 responses changed wording, because the batched GPU kernels are not bit-deterministic here, so differences of a few refusals are not significant. The refusal count is a keyword match. Read by hand, in the run above, 4 of the 21 flagged responses decline or reframe the request; the other 17 comply and include a warning ("illegal", "disclaimer") that matches a marker.
Why thinking off. The chat template turns thinking on by default, and Heretic scores whatever the template produces. With thinking on, the 100 scored tokens are the start of the thinking trace rather than the answer, and the marker list matches a trace that discusses safety before it complies. Measured the same way with thinking on: original 90/100, this model 40/100, KL 0.4357 (median 0.0102: a few prompts at a near-tied first token dominate the mean). The thinking-off numbers measure the answer and are the ones used on all cards in this family.
The abliteration this model was made from reports 1/100 refusals at KL 0.0495 against the base model, measured by its author with Heretic; that measurement and this one are not directly comparable.
Standard capability benchmarks were not run.
How it was made
The refusal behaviour was removed by a weight-delta transplant, not by a fresh search. ThinkingCap is a fine-tune of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16, and trohrbaugh published an abliteration of that base model, NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16-heretic, made with Heretic (directional ablation of attention o_proj, Mamba out_proj and MoE down_proj). For every tensor the abliteration changed, the difference abliterated minus base was added to ThinkingCap at scale 1.0, in fp32, then rounded to bf16:
ThinkingCap + 1.0 × (abliterated − base)
2,343 of 6,513 tensors changed: 6 attention o_proj, 15 Mamba out_proj, 2,304 routed-expert down_proj and 18 shared-expert down_proj. Every other tensor, the MTP head included, is ThinkingCap's, bit for bit. ThinkingCap differs from the base model in only 70 tensors, so the edited weights mostly start from the same values the abliteration was computed on.
What is preserved
- Thinking-length behaviour: ThinkingCap's weights outside the edited projections and its chat template are unmodified. Its token-efficiency numbers were not re-benchmarked on this model.
- MTP head: present and unmodified.
- Tool calling: same chat template and parser flags as the original; not separately tested.
Limitations and intended use
- The model complies with requests the original declined. It has no safety filter; you are the safety layer. Run it behind your own policy controls, for research, red-teaming, creative work, or agent pipelines where over-refusal was the problem.
- 21/100 harmful prompts still match a refusal marker with thinking off, most of them answers with a warning attached; with thinking on, the trace may still discuss policy before the answer complies. Test in the mode you use.
- All limitations of the original ThinkingCap and of Nemotron 3.5 Lightning apply.
License
OpenMDW License Agreement 1.1, inherited from ThinkingCap and the NVIDIA base model (see LICENSE).
Credits
bottlecapai (ThinkingCap) · trohrbaugh (the abliteration this transplant uses) · p-e-w/heretic (evaluation protocol and the method behind the abliteration) · NVIDIA (Nemotron 3.5 Lightning)