library_name: ninfer
pipeline_tag: image-text-to-text
inference: false
license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
tags:

• ninfer
• groupwise-int
• qwen3.8
• qwen3_5
• abliterated
• uncensored
• multimodal
• mtp
• speculative-decoding
• blackwell
• cuda

Qwen3.8-27B abliterated — groupwise-int for NInfer

A groupwise-int artifact of huihui-ai/Huihui-Qwen3.8-27B-abliterated,
packaged for the NInfer engine.

This is the refusal-removed counterpart to the official
neroued/Qwen3.8-27B-NInfer artifact, using the
same weights profile. One file carries the complete container — Text, Vision, MTP, the optimized
proposal head, tokenizer, chat template, generation config and media-processor resources.

qwen3_8_27b_huihui_abliterated.ninfer
  18,210,531,328 bytes
  sha256 8c9f9d67a07ac97506978f6db6695d8074f78dec0fb80c4a85a8fb6fbedd7f03
  identity  qwen3.8-27b / groupwise-int      container v2      1124 objects

There is also an NVFP4 build
of this same checkpoint. It is ~35 % faster on speculative decode and 1.6–2× on prefill, for 3.1 GiB
more VRAM. Prefer NVFP4 unless VRAM is tight. This profile's advantages are a smaller footprint
(15.9 GiB resident vs 19.0) and a dramatically simpler build — one command, no external quantizer,
no calibration data, and no patch of any kind.

Why this exists

Qwen3.8-27B is the king of the local models right now, and the great work Neroued has done with the
NInfer engine runs it extremely quickly on Blackwell. There has
also been a lot of work on obliterating and otherwise liberating models, including
Pliny's OBLITERATUS suite.

Some abliterated models have been "ninferized" already — see Related work
below. Two things I hadn't seen:

• the walkthrough/recipe to do it yourself
• Pliny's OBLITERATUS Qwen3.8-27B, ninferized

Both are offered here. The recipe lives in the
NVFP4 repo —
AGENT.md, SUPPORTED.md and scripts/. Under $5 of RunPod time converts a compatible
checkpoint; measured runs came in at $1.69 and $3.30.

What it is, precisely

Format conversion only. No fine-tuning, no re-abliteration, no behavioural editing. All refusal
characteristics are inherited unchanged from the upstream huihui-ai checkpoint, which removes the
refusal direction by weight editing (per
remove-refusals-with-transformers,
the Arditi et al. 2024 family) on layers 18–51 only — layers 0–17 and 52–63, the vision tower and
the MTP head are untouched.

Quantization is round-to-nearest group quantization performed by NInfer's own converter. Nothing
external is involved.

Numeric allocation

Layouts: contiguous-le-v1 679, row-split-k128-v1 439. All grouped-integer tensors use
MAXABS_F16_RECIP_RNE_V1.

Structurally identical to the official groupwise-int artifact — same 1124 objects, same per-format
and per-layout counts, same file size to the byte. Only the values differ.

Requirements

• Blackwell GPU, compute capability 12.0 (sm_120a). NInfer's build rejects any other architecture.
• ~15.9 GiB VRAM for weights; a 32 GB card leaves ~13.6 GiB for KV cache.
• Stock NInfer — no engine patch required.

Quick start

ninfer qwen3_8_27b_huihui_abliterated.ninfer \
  --prompt "Explain group quantization in three sentences." \
  --max-context 16384 --max-new 256 \
  --spec mtp --draft-tokens 5 --lm-head-draft

Serving:

ninfer-serve qwen3_8_27b_huihui_abliterated.ninfer \
  --model-id qwen3.8-27b-huihui-abliterated \
  --max-context 262144 --kv-capacity auto --max-concurrency 1 \
  --spec mtp --draft-tokens 5 --lm-head-draft \
  --preserve-thinking --vision \
  --host 0.0.0.0 --port 8080

--model-id is recommended: the on-disk identity is qwen3.8-27b/groupwise-int, identical to the
official artifact, so without it the two are indistinguishable to any client.

Measured

Same GPU, identical flags, --max-context 16384, greedy.

Scope of validation — read this

Verified: the artifact loads on stock NInfer; MTP speculative decoding engages at 86.2 % acceptance
with zero fallbacks; thinking mode works; output is coherent and correct on reasoning, code and
explanation prompts; the container is structurally byte-for-byte equivalent to the official artifact;
the source checkpoint is intact (1199 tensors, 0 non-finite values).

Not run: no capability benchmark, no fidelity comparison against BF16, no refusal-rate measurement.
No HumanEval+/GPQA numbers are claimed because none were produced. Abliteration has a documented
capability cost and this artifact inherits it, unquantified.

Verify

sha256sum -c SHA256SUMS
python3 -m tools.artifact.inspect qwen3_8_27b_huihui_abliterated.ninfer

Expect identity qwen3.8-27b/groupwise-int, objects 1124 (1118 tensors, 6 resources),
file_bytes 18210531328.

Reproducing it

Genuinely one command. No external quantizer, no calibration set, no patch — patches/ for this
profile is empty, and upstream NInfer is used unmodified at
a140e7ae82a11ed2f370a4d8f2cc16268a3790b8:

python -m tools.convert.qwen3_8_27b.convert \
  --model /path/to/huihui-abliterated \
  --out   out/qwen3_8_27b.ninfer \
  --device cpu

~12 minutes. Expect this first line, which is the whole validation story:

preflight complete: 1124 objects, 1199 source tensors, device=cpu

That single message is the converter accepting an abliterated checkpoint through every gate it has:
SHA-256 over six frontend resources, architecture member checks, shape and dtype validation across all
1199 source tensors, and a hardcoded inventory assertion.

Memory. Peak RSS is 18.2 GB. quantize_matrix upcasts to float32 and allocates several
full-size temporaries — measured at 6.1× the bf16 tensor size — and both text/token_embedding
and text/output_head are [248320, 5120] = 2.54 GB each. With less RAM than that the converter is
OOM-killed on object 7 of 1124. --device cuda also works and needs ~18 GB VRAM.

Source choice matters. The converter's only canonical-source gate is a SHA-256 check over six
frontend files. Of four abliterated candidates with a complete tensor set, only huihui-ai ships all
six byte-identical to canonical (the others match 0–3 of 6), so it converts with no patch and no file
substitution. Full candidate comparison and the RunPod cost breakdown are in the
NVFP4 repo's REPLICATION.md.

Related work — other NInfer abliterated artifacts

Other people got here first, and their artifacts are also good. Use whichever fits — this list is
here so you can find them.

Worth noting three are built on
orcarouter/Qwen3.8-27B-Uncensored,
which is gated.

As far as I can tell, none of the above ship a conversion recipe. That is what this repo adds.

Licence

Apache-2.0, inherited from the whole chain — Qwen/Qwen3.8-27B (Copyright 2026 Alibaba Cloud),
huihui-ai/Huihui-Qwen3.8-27B-abliterated, and Neroued/ninfer are all Apache-2.0. LICENSE is the
file distributed with the upstream checkpoint, notice intact (§4(c)); NOTICE carries the attribution
chain and statement of changes (§4(b)). No upstream component ships a NOTICE file, so §4(d) does not
apply. Unlike the NVFP4 build, this profile involved no calibration data and no modification to
NInfer whatsoever.

"Qwen" is a trademark of Alibaba Cloud. Unofficial community derivative, not endorsed by or affiliated
with Alibaba Cloud, huihui-ai, or the NInfer project.

A description of what the licences say, not legal advice.

Intended use and limitations

This is an uncensored model: the upstream abliteration removes the refusal direction, so it will
attempt requests a stock instruction-tuned model declines. Published for research, evaluation and
local deployment where that is understood and wanted. Anyone deploying it is responsible for their own
safeguards and for compliance with the Apache-2.0 licence and applicable law. Refusal behaviour and any
capability regression originate entirely upstream — this repository changes format, not behaviour.

Credit for the abliteration to @huihui-ai; the base model to
Qwen; the engine and artifact contract to
@Neroued.