← back to catalog · registered 2026-11-17 19:04

acidkill/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-NPU2

acidkill Nemotron GGUF multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/acidkill%2FNVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-NPU2"
Response includes
  • classification m8
  • files 15
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
38
Likes
0
Model age
4d ago
created 2026-11-17
Downloads over time
Now 38 → from 0 ↑ 0%
0142842 0 on Nov 17 38 on Nov 22 Nov
Nov 17 → Nov 22 · 6 snapshots · spans 5 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
other
Tags
gemma4_text npu q4nx openflowlm gemma4 nemotron abliterated experimental image-text-to-text conversational base_model:AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF base_model:quantized:AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF

Related

Total size
0 B
Files
15
Quantizations
1
Registered
2026-11-17 19:04
Last updated on HF
2026-11-17 19:03

Files by quantization

Auxiliary files 15 files 8.48 GB
model.q4nx 6.65 GB f62addbc download
vision_weight.q4nx 912 MB d04d4861 download
audio_weight.q4nx 912 MB e2f38214 download
tokenizer.json 30.7 MB cc8d3a0c download
source-tokenizer-provenance.json 13.1 MB 6e16a29f download
benchmarks.json 71.7 KB ecb9ec03 download
chat_template.jinja 19.7 KB e4ac6701 download
LICENSE.Apache-2.0 11.1 KB d6456956 download
README.md 6.81 KB bd0f0f9e download
config.json 6.60 KB b2255657 download
LICENSE 2.56 KB ec297ac5 download
tokenizer_config.json 2.09 KB d55e0714 download
SHA256SUMS 1.12 KB cb19862b download
NOTICE 705 B f8797b2c download
.gitattributes 163 B c8fa10d5 download

README current version from Hugging Face


license: other
license_name: openmdw-1.1
license_link: LICENSE
author: acidkill
base_model: AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:

  • npu
  • q4nx
  • openflowlm
  • gemma4
  • nemotron
  • abliterated
  • experimental

NVIDIA Nemotron 3.5 Bolt E4B Uncensored — NPU2

Experimental OpenFlowLM conversion for AMD XDNA2, derived from AtomicChat's baked AD-Q6_K release. NVIDIA post-trained the Gemma4 E4B base; AtomicChat performed the refusal-direction edit and source GGUF quantization.

Known limitation: the original small frozen evaluation scored 11/14 on both NPU and GPU, failing its automatic-tool-choice gate. Both functions passed complete roundtrips on both backends after an explicit agent system instruction. Those supplemental results are separate; the original failed gate is retained below and in benchmarks.json.

NPU2 refers to the AMD XDNA2 target. This package uses native mixed Q4_1/int8/BF16 storage, including int8 token/per-layer embeddings; it is not a 2-bit model. Source GGUF tensors contain 7,518,069,290 parameters. E4B is the upstream model's effective-size naming.

Run

Tested with OpenFlowLM0.1.0 including its existing closed gemma4e engine and Gemma4-E4B-IT-NPU2 kernels. Engine/kernel binaries are provided by OFLM, not this repository. No new kernels were built for this conversion.

oflm add acidkill/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-NPU2 --tag nemotron-bolt:e4b-uncensored --family gemma4e --xclbin-from Gemma4-E4B-IT-NPU2
oflm serve nemotron-bolt:e4b-uncensored --host 127.0.0.1 --port 52626 --ctx-len 4096 --pmode performance
curl http://127.0.0.1:52626/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"nemotron-bolt:e4b-uncensored","messages":[{"role":"user","content":"Explain RAM and VRAM."}],"think":false,"chat_template_kwargs":{"enable_thinking":false},"max_tokens":256}'

When sending tools, use an explicit agent policy. The additional two-function test used this exact system instruction:

You must use the provided tools to obtain results. Do not calculate or invent tool results yourself. Do not give a final answer until the tool response is available.

The original NVIDIA template embedded in AtomicChat's source GGUF is preserved verbatim, including enable_thinking and optional conversational_mode. No persona or tool-forcing instruction was inserted into the stored template. All local measurements disabled thinking; thinking-on quality is untested.

Size and memory

Weight file Bytes
model.q4nx 7,142,621,252
vision_weight.q4nx 956,326,592
audio_weight.q4nx 956,326,424

Total weights: 9.055 GB / 8.43 GiB. The measured NPU service cgroup peak was 11.65 GB, with zero swap at the final snapshot on a 32GB machine, context4096. Runtime memory exceeds file size; other RAM sizes are not independently certified.

The native language/vision/audio layouts matched the pinned Gemma4 E4B skeleton exactly: 849/1411/1411 tensor keys, shapes and dtypes. Raw floating tensors scanned finite. Tokenizer validation matched all262144 vocabulary IDs and514906 merges, with source BOS2 and EOS1/106. Original tokenizer metadata/types/template are retained in the provenance sidecar. This is a lossy conversion, not numerical-parity certification.

Local tests

Ryzen AI7 350 / Radeon860M / Krackan1 XDNA2, 32GB RAM, Linux7.2.15-2-omarchy; OFLM0.1.0, explicit --pmode performance; LlamaStash0.11.0, llama.cpp c1304d7 on ROCm0.

Frozen evaluation NPU2 GPU AD-Q6_K
Text 8/8 8/8
Images 3/4 3/4
Automatic tool choice 0/2 0/2
Overall 11/14 11/14
Configured frozen gate failed failed

The OCR fixture expected NPU2; NPU returned NPUZ, GPU returned NFU2. In the two basic tool cases both backends answered directly instead of calling the function. With the explicit system policy above, 2/2 additional tool roundtrips passed on each backend, including correct arguments and the final answer after the mock tool response. This demonstrates a tested prompt configuration, not broad agent reliability.

Actual NPU execution was observed through the accelerator FD and ten XRT contexts: 20568 completed commands, zero reported errors at the recorded snapshot.

Performance comparison

Same profile text, 1237 input /128 generated tokens per sample on both backends; temperature0, thinking off, cache reuse disabled for these profiles. Medians of3; separate streamed TTFT measurement n=1:

Text profile NPU2 GPU AD-Q6_K
Prefill 483.11 tok/s 250.26 tok/s
Generation 12.47 tok/s 18.18 tok/s
End-to-end 12.923 s 12.034 s
Client TTFT 2.522 s 4.937 s

GPU reference used the same full AD-Q6_K input plus AtomicChat's original F16 projector. LlamaStash admission remained enabled; loading used --no-mmap --fit off. Logs confirmed43/43 layers offloaded, a4165.69MiB ROCm model buffer and3536.00MiB host model buffer. Embedding tables can reside on the host even with all decoder layers offloaded.

GPU profile cached-token counts were zero; NPU logs showed misses. Warm arithmetic/image latency is also recorded in benchmarks.json under each backend's normal caching. These backends and quantizations differ; the small measurements do not isolate hardware speed or establish broad quality equivalence. Both servers were stopped after testing.

Provenance and limits

  • Baked language source: AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF @ 43409f2bb0e36573ed3990e85f9f47e81021c3c5, full AD-Q6_K file.
  • Vision/audio input: AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF @ 65532257bfff429fabede8a90c951a74247d4c7d, original F16 projector.
  • Native tokenizer/configuration skeleton: Atomic-Germ/Gemma4-E4B-IT-NPU2 @ 1621cd87cc7ef9fdd89f224092655ba2c1609c43.

All complete inputs were checked against pinned hashes; final file hashes are in SHA256SUMS. The configured context was4096; the metadata ceiling follows the source64k limit.

Audio weights were converted and structurally checked; audio inference was not tested. Long context, advanced schemas, broad coding/knowledge accuracy, refusal behavior and thinking-on acceptance were not evaluated. See AtomicChat's original card for the edit's own published limitations. A fresh public download/install was explicitly waived by the publisher; only public visibility was verified after upload.

License

Model materials: OpenMDW-1.1, included as LICENSE. Underlying Gemma materials carry Apache2.0, included as LICENSE.Apache-2.0. Retain the included NOTICE and upstream origin/license notices. See the source repository and NVIDIA's upstream model card for lineage.

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-17 Upload Nemotron Bolt E4B NPU2 weights, model card and benchmarks 7c079c5 6.8 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration