license: apache-2.0
library_name: Model Optimizer
pipeline_tag: image-text-to-text
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
tags:
- abliterated
- uncensored
- quantized
- modelopt
- nvfp4
- fp8
- qwen3.8
- sglang
- blackwell
Huihui-Qwen3.8-27B-abliterated-NVFP4-BF16-LMHead
NVFP4 quantization of
huihui-ai/Huihui-Qwen3.8-27B-abliterated,
an abliterated version of Qwen/Qwen3.8-27B. The MLPs are
NVFP4 (W4A4), attention and Gated DeltaNet projections are FP8, and lm_head stays in BF16.
[!WARNING]
Uncensored model. Abliteration removed the base model's refusal behaviour. The model has no
safety guardrails and will answer requests the original model declines. You are responsible for
how you use it and for any safeguards your application needs.
Model details
| Base model | Qwen/Qwen3.8-27B → huihui-ai/Huihui-Qwen3.8-27B-abliterated (revision 739e3c5b) → this quantization |
| Architecture | Dense hybrid Gated DeltaNet vision-language model, 27B parameters, 64 layers, native context 262,144 tokens |
| Input / output | Text, image and video in; text out |
| Weights | 23.75 GB (3 safetensors shards) |
| Hardware | NVIDIA Blackwell GPU (native NVFP4 kernels) |
| License | Apache-2.0, inherited from Qwen3.8-27B and the huihui-ai release |
Quantization
| Component | Format |
|---|---|
MLP gate_proj / up_proj / down_proj (192 layers) |
NVFP4 W4A4, group size 16 |
self_attn.q/k/v/o_proj (64 layers) |
FP8 W8A8 |
linear_attn.in_proj_qkv / in_proj_z / out_proj (144 layers) |
FP8 W8A8 |
| KV cache | FP8 |
lm_head, embeddings, linear_attn.in_proj_a/b, conv1d, vision tower, MTP head |
BF16 (unquantized) |
- Tool: NVIDIA ModelOpt 0.47.0 (
examples/hf_ptq/hf_ptq.py), transformers 5.12.1 - Calibration: 1,024 samples from
abisee/cnn_dailymail(train), sequence length 512,maxcalibration - Recipe:
recipe/qwen38_w4a4_nvfp4-fp8_attn-bf16_head-kv_fp8_cast.yaml
How to use
Serve with SGLang:
sglang serve --trust-remote-code \
--model-path iQlusion/Huihui-Qwen3.8-27B-abliterated-NVFP4-BF16-LMHead \
--reasoning-parser qwen3 --tool-call-parser qwen3_coder \
--host 0.0.0.0 --port 30000
Query the OpenAI-compatible endpoint:
curl -s http://localhost:30000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "default", "messages": [{"role": "user", "content": "What is 17 * 23?"}]}'
Thinking mode is on by default; pass "chat_template_kwargs": {"enable_thinking": false} in the
request to turn it off. The checkpoint includes the base model's MTP head, so SGLang's MTP speculative
decoding (--speculative-algorithm EAGLE) can be used; see the
SGLang Qwen3.8-27B cookbook for
per-GPU flags.
Intended use and limitations
- Intended use: research and self-hosted deployments where the base model's refusals are unwanted.
- Out of scope: any use that breaks the law or harms others, and production use without your own
content safeguards. - Inherited behaviour: the model can produce inaccurate, biased or harmful text, as the base model
can, and it no longer refuses. - Calibration data: calibration used English news text only. Quality on code, maths and other
languages may drop more than on English prose. - Vision: the vision tower is unquantized, but the language layers were calibrated on text only.
- MTP head: copied unchanged from the huihui-ai checkpoint. Speculative decoding never changes
outputs; its acceptance rate may differ from the stock model's.
Evaluation
No benchmark results are published for this checkpoint yet. The export was checked layer by layer:
all 192 MLP projections are NVFP4 (group size 16), all 208 attention and Gated DeltaNet projections
are FP8, lm_head is BF16, and the 15 MTP and 333 vision tensors are present.
Credits
- Qwen: base model
- huihui-ai: abliteration
- NVIDIA ModelOpt: quantization tooling
- Abliteration technique: Arditi et al., Refusal in Language Models Is Mediated by a Single
Direction (2024)