language: [en, es, fr, de, it, ja]
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
pipeline_tag: text-generation
tags:
- nemotron
- nemotron-3.5
- nemotron_h
- mamba
- moe
- gguf
- abliterated
- heretic
- uncensored
- decensored
- text-generation
- roleplay
base_model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16
base_model_relation: quantized
quantized_by: mlasli
Nemotron-3.5-Lightning-30B-A3B Heretic-Abliterated (Q4_K_M GGUF)
GGUF quantization of
mlasli/Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-BF16
— NVIDIA-Nemotron-3.5-Lightning-30B-A3B (31.6B total / 3B active)
with its refusal direction removed via Heretic.
What this is for: the same abliterated hybrid Mamba-MoE model, quantized to 24.3 GB so it runs
locally via llama.cpp.
- Architecture:
nemotron_h_moe(requires llama.cpp build b10326+) - Quantization: Q4_K_M
- File size: 24.3 GB
- Smoke-tested locally before upload (loads + coherent output on
llama-cli).
Results
| Refusals | Compliance | KL Divergence | Trials |
|---|---|---|---|
| 0% | 100% | 0.0397 | 200 |
Independent eval of the merged BF16 model (50 harmful-behavior prompts). The automated Zou
keyword detector false-positives on words like "illegal"/"unethical" appearing inside compliant
answers; manual review found 0 genuine refusals.
Usage
llama-cli -m Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-Q4_K_M-GGUF.gguf \
-cnv -st -p "What is 2+2?"
Ollama
Create a Modelfile:
FROM ./Nemotron-3.5-Lightning-30B-A3B-Heretic-Uncensored-Q4_K_M-GGUF.gguf
Then:
ollama create nemotron-3.5-30b-heretic-q4_k_m
ollama run nemotron-3.5-30b-heretic-q4_k_m
Abliteration removes safety alignment. Use responsibly and in accordance with your local laws
and the upstream NVIDIA Open Model License.