license: apache-2.0
base_model:
- huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
tags: - ninfer
- qwen3.8
- abliterated
- uncensored
- rtx-3090
- image-text-to-text
Huihui-Qwen3.8-27B-abliterated, NInfer v3 artifact
A NInfer v3 artifact of
huihui-ai/Huihui-Qwen3.8-27B-abliterated,
the abliterated Qwen3.8-27B with the lowest reported divergence from the base checkpoint among the
widely used releases (the first fifteen layers are untouched; independent KL divergence
measurements sit around 0.008) and a working MTP head.
Built for the RTX 3090 line of NInfer
(iamwavecut/ninfer-3090, branch franken/v0.11,
which is ashalliants' v0.11.0 plus that fork's production fixes).
What is inside
| component | representation |
|---|---|
| text layers (64) | official qwen3_8_27b recipe: Q4/Q5 grouped projections, Q8 vocabulary embedding and output head |
| vision tower | as the recipe assigns it (the checkpoint's own model.visual.* weights) |
| MTP head | the checkpoint's own mtp.* weights, Q8 |
| DFlash2 adapter | z-lab/Qwen3.8-27B-DFlash2, trained on the vanilla model; applied unchanged (the target verifies every draft, so it only affects acceptance) |
| proposal head | the indexed 131,072-row draft head for --lm-head-draft |
| chat template | NInfer's pinned qwen3_8.jinja |
Conversion command (from the fork's tree, CUDA device):
python3 -m tools.convert \
--model Huihui-Qwen3.8-27B-abliterated \
--recipe qwen3_8_27b \
--source dflash2=Qwen3.8-27B-DFlash2 \
--components text,vision,mtp,dflash2 \
--resource chat_template.jinja=tools/chat_templates/qwen3_8.jinja \
--proposal \
--name qwen3.8-27b-abliterated \
--out Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer
SHA256SUMS lists the checksum of every file.
Running
Any NInfer build that reads v3 artifacts (v0.11.0 or later). A single RTX 3090 serves the full
198,400-token window with vision and MTP:
ninfer-serve Huihui-Qwen3.8-27B-abliterated-ninfer-v3.ninfer --model-id qwen3.8-27b \
--max-context 198400 --kv-capacity 198400 --kv-dtype rk8v4 --gdn-state-fp16 \
--spec mtp --draft-tokens 3 --lm-head-draft \
--vision --vision-residency overlay --vision-max-merged 12288
--spec dflash2 --draft-tokens 7 selects the DFlash2 backend instead of MTP.
Caveats
This is an abliterated model: refusals are removed at the checkpoint level, and the usual
hesitation before destructive actions goes with them. Put the guardrails in the tool layer, not in
the model. rk8v4 is a lossy KV format (about +0.08 % perplexity against bf16 on the vanilla
checkpoint) chosen for the context window; --kv-dtype int8 or bf16 trade window for fidelity.