license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.5-4B-abliterated
base_model_relation: quantized
pipeline_tag: text-generation
tags:

• tokforge
• on-device
• npu
• qualcomm
• qnn
• hexagon
• mnn
• qwen3.5
• abliterated

Published by TokForge for on-device inference.

TokForge

• Website: https://tokforge.ai
• Discord: https://discord.gg/Acv3CBtfVm
• Google Play: https://play.google.com/store/apps/details?id=dev.tokforge
• iOS TestFlight: https://testflight.apple.com/join/jnufjzRr

Runs on-device in the TokForge app.

Qwen3.5 4B Abliterated NPU prompt pack (catalog pair)

An NPU pack for Qwen3.5 4B Abliterated in TokForge on Snapdragon phones. The phone's NPU reads long
prompts; the CPU writes every reply with the Qwen3.5 4B Abliterated download you already have. TokForge downloads this pack only
when you turn the NPU option on for this model, and it checks the pack on your phone before it uses it.

Original model by the Qwen team; abliterated (fewer refusals) by
huihui-ai. This pack is exported from TokForge's MNN
conversion of that model and only changes the runtime format for the phone's NPU.

What is in it

QNN context binaries (the model's layers compiled for the Hexagon NPU) for two chips, plus the small MNN graph and
configs that drive them. A phone downloads only the folder for its own chip.

Each chip folder holds 49 context binaries (qnn/graph0.bin to qnn/graph48.bin), qnn/llm.mnn (the MNN graph
that drives them), qnn/llm_config.json and config_qnn.json. Qwen3.5-4B-uncensored-MNN-catalogpair/MANIFEST.json lists every file per chip
with its sha256 and size, and the exact download the pack pairs with. SHA256SUMS covers every file in this repo.

Pairs with one exact download

The pack works only on top of darkmaniac7/Qwen3.5-4B-uncensored-MNN at commit
9416259dd5ed (branch v2, the TokForge MNN conversion of
huihui-ai/Huihui-Qwen3.5-4B-abliterated). TokForge matches the two by file hashes:

With any other download the pack is not used. The pack has no weight file of its own: the NPU weights are inside
the compiled graphs, and the token embeddings are read from the paired download's llm.mnn.weight.

What runs where

• NPU: the 49 compiled graphs, which hold the projections, the MLP blocks and most of the norms of all 32 layers.
• CPU, in the same prompt pass: the 24 linear-attention cores and the 8 full-attention cores with their
  key/value cache, a few small norms (their constants are inside qnn/llm.mnn) and the token embedding lookup.
• CPU, with your download: the end of the prompt that does not fill a whole 128-token chunk, and every token of
  the reply. The pack has no output layer, so the reply always comes from the model you downloaded.
• Short prompts stay on the CPU. If the NPU stops during a reply, the CPU finishes it.

How it was built

Exported from the paired download's own int4 weights: MNN's loader dequantizes them, and MNN llmexport (TokForge's
MNN branch, commit d729273a91cd) writes 8-bit per-channel symmetric weights with fp16 activations, feeds
the rotary tables as fp32 inputs and cuts the graph before the output layer. compilefornpu splits the 32 layers into
49 NPU graphs at 24 linear-attention breaks and 40 scalar breaks, each with a 128-row chunk shape and a 1-row shape.
The Qualcomm AI Engine Direct SDK from QAIRT 2.40.0.251030 compiles every graph into one context binary per
chip with the strict HTP options (optimization level 3, fp16 relaxed precision, high-precision sigmoid, advanced
activation fusion off, 8 MB VTCM, 4 HVX threads, weight sharing, and extended UDMA on V81 only, since V75 does not accept it). The build stops on any difference between a graph's declared inputs and outputs and what QNN created (98
of 98 checks pass on each chip), and every context binary is read back against its declaration (49 of 49 on each
chip). The NPU reads the prompt in 128-token chunks. Source training weights were not edited.

Usage with TokForge

1. Install TokForge on a phone with one of the chips above.
2. Download Qwen3.5 4B Abliterated from the app's model list.
3. Turn on the NPU option for the model when TokForge offers it. TokForge downloads the pack for your chip, checks
   every file by sha256, and compares the NPU with the CPU on your phone before it uses the NPU.

The Qualcomm runtime libraries are not in this repo; TokForge ships them inside the app.

Limitations

• Snapdragon 8 Elite Gen 5 and Snapdragon 8 Gen 3 only. On other chips TokForge reads prompts on the CPU.
• The NPU part needs about 6.3 GB of memory while the model is loaded; TokForge turns it on only when the phone has
  room for it.
• Runtime files for TokForge and MNN, not a Transformers checkpoint.
• The model can still get facts wrong. Check important answers.

License and attribution

Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0), as the original model, the abliterated model and the MNN conversion. All
source authors keep their copyrights; no endorsement is claimed.

Credits

• Huihui-Qwen3.5-4B-abliterated by huihui-ai (Hugging Face), an abliterated Qwen3.5-4B. Used under the Apache-2.0 license.
• Qwen3.5-4B by the Qwen team (Alibaba Cloud) (Hugging Face). Used under the Apache-2.0 license.
• The MNN conversion darkmaniac7/Qwen3.5-4B-uncensored-MNN (branch v2) by TokForge, whose int4 weights this pack is exported from. Apache-2.0.
• Exported with llmexport and run by MNN by Alibaba. Apache-2.0.
• Compiled for the Hexagon NPU with the Qualcomm AI Engine Direct SDK from the Qualcomm AI Runtime (QAIRT) 2.40.0.251030 by Qualcomm. This pack contains compiled model graphs only, no Qualcomm runtime libraries.