license: other
license_name: openmdw-1.1
license_link: LICENSE
author: acidkill
base_model: AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:
- npu
- q4nx
- openflowlm
- gemma4
- nemotron
- abliterated
- experimental
NVIDIA Nemotron 3.5 Bolt E4B Uncensored — NPU2
Experimental OpenFlowLM conversion for AMD XDNA2, derived from AtomicChat's baked AD-Q6_K release. NVIDIA post-trained the Gemma4 E4B base; AtomicChat performed the refusal-direction edit and source GGUF quantization.
Known limitation: the original small frozen evaluation scored 11/14 on both NPU and GPU, failing its automatic-tool-choice gate. Both functions passed complete roundtrips on both backends after an explicit agent system instruction. Those supplemental results are separate; the original failed gate is retained below and in benchmarks.json.
NPU2 refers to the AMD XDNA2 target. This package uses native mixed Q4_1/int8/BF16 storage, including int8 token/per-layer embeddings; it is not a 2-bit model. Source GGUF tensors contain 7,518,069,290 parameters. E4B is the upstream model's effective-size naming.
Run
Tested with OpenFlowLM0.1.0 including its existing closed gemma4e engine and Gemma4-E4B-IT-NPU2 kernels. Engine/kernel binaries are provided by OFLM, not this repository. No new kernels were built for this conversion.
oflm add acidkill/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-NPU2 --tag nemotron-bolt:e4b-uncensored --family gemma4e --xclbin-from Gemma4-E4B-IT-NPU2
oflm serve nemotron-bolt:e4b-uncensored --host 127.0.0.1 --port 52626 --ctx-len 4096 --pmode performance
curl http://127.0.0.1:52626/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"nemotron-bolt:e4b-uncensored","messages":[{"role":"user","content":"Explain RAM and VRAM."}],"think":false,"chat_template_kwargs":{"enable_thinking":false},"max_tokens":256}'
When sending tools, use an explicit agent policy. The additional two-function test used this exact system instruction:
You must use the provided tools to obtain results. Do not calculate or invent tool results yourself. Do not give a final answer until the tool response is available.
The original NVIDIA template embedded in AtomicChat's source GGUF is preserved verbatim, including enable_thinking and optional conversational_mode. No persona or tool-forcing instruction was inserted into the stored template. All local measurements disabled thinking; thinking-on quality is untested.
Size and memory
| Weight file | Bytes |
|---|---|
model.q4nx |
7,142,621,252 |
vision_weight.q4nx |
956,326,592 |
audio_weight.q4nx |
956,326,424 |
Total weights: 9.055 GB / 8.43 GiB. The measured NPU service cgroup peak was 11.65 GB, with zero swap at the final snapshot on a 32GB machine, context4096. Runtime memory exceeds file size; other RAM sizes are not independently certified.
The native language/vision/audio layouts matched the pinned Gemma4 E4B skeleton exactly: 849/1411/1411 tensor keys, shapes and dtypes. Raw floating tensors scanned finite. Tokenizer validation matched all262144 vocabulary IDs and514906 merges, with source BOS2 and EOS1/106. Original tokenizer metadata/types/template are retained in the provenance sidecar. This is a lossy conversion, not numerical-parity certification.
Local tests
Ryzen AI7 350 / Radeon860M / Krackan1 XDNA2, 32GB RAM, Linux7.2.15-2-omarchy; OFLM0.1.0, explicit --pmode performance; LlamaStash0.11.0, llama.cpp c1304d7 on ROCm0.
| Frozen evaluation | NPU2 | GPU AD-Q6_K |
|---|---|---|
| Text | 8/8 | 8/8 |
| Images | 3/4 | 3/4 |
| Automatic tool choice | 0/2 | 0/2 |
| Overall | 11/14 | 11/14 |
| Configured frozen gate | failed | failed |
The OCR fixture expected NPU2; NPU returned NPUZ, GPU returned NFU2. In the two basic tool cases both backends answered directly instead of calling the function. With the explicit system policy above, 2/2 additional tool roundtrips passed on each backend, including correct arguments and the final answer after the mock tool response. This demonstrates a tested prompt configuration, not broad agent reliability.
Actual NPU execution was observed through the accelerator FD and ten XRT contexts: 20568 completed commands, zero reported errors at the recorded snapshot.
Performance comparison
Same profile text, 1237 input /128 generated tokens per sample on both backends; temperature0, thinking off, cache reuse disabled for these profiles. Medians of3; separate streamed TTFT measurement n=1:
| Text profile | NPU2 | GPU AD-Q6_K |
|---|---|---|
| Prefill | 483.11 tok/s | 250.26 tok/s |
| Generation | 12.47 tok/s | 18.18 tok/s |
| End-to-end | 12.923 s | 12.034 s |
| Client TTFT | 2.522 s | 4.937 s |
GPU reference used the same full AD-Q6_K input plus AtomicChat's original F16 projector. LlamaStash admission remained enabled; loading used --no-mmap --fit off. Logs confirmed43/43 layers offloaded, a4165.69MiB ROCm model buffer and3536.00MiB host model buffer. Embedding tables can reside on the host even with all decoder layers offloaded.
GPU profile cached-token counts were zero; NPU logs showed misses. Warm arithmetic/image latency is also recorded in benchmarks.json under each backend's normal caching. These backends and quantizations differ; the small measurements do not isolate hardware speed or establish broad quality equivalence. Both servers were stopped after testing.
Provenance and limits
- Baked language source:
AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-Uncensored-GGUF@43409f2bb0e36573ed3990e85f9f47e81021c3c5, full AD-Q6_K file. - Vision/audio input:
AtomicChat/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF@65532257bfff429fabede8a90c951a74247d4c7d, original F16 projector. - Native tokenizer/configuration skeleton:
Atomic-Germ/Gemma4-E4B-IT-NPU2@1621cd87cc7ef9fdd89f224092655ba2c1609c43.
All complete inputs were checked against pinned hashes; final file hashes are in SHA256SUMS. The configured context was4096; the metadata ceiling follows the source64k limit.
Audio weights were converted and structurally checked; audio inference was not tested. Long context, advanced schemas, broad coding/knowledge accuracy, refusal behavior and thinking-on acceptance were not evaluated. See AtomicChat's original card for the edit's own published limitations. A fresh public download/install was explicitly waived by the publisher; only public visibility was verified after upload.
License
Model materials: OpenMDW-1.1, included as LICENSE. Underlying Gemma materials carry Apache2.0, included as LICENSE.Apache-2.0. Retain the included NOTICE and upstream origin/license notices. See the source repository and NVIDIA's upstream model card for lineage.