license: apache-2.0
language:
- en
- tr
base_model: - huihui-ai/Huihui-Qwen3.8-27B-abliterated
- Qwen/Qwen3.8-27B
tags: - qwen3.8
- qwen3
- uncensored
- abliterated
- nvfp4
- fp4-kv-cache
- ninfer
- rtx-5090
- blackwell
- gated-deltanet
- long-context
- dflash2
Huihui‑Qwen3.8‑27B‑Uncensored — ready‑to‑run NInfer artifact (RTX 5090)
A prebuilt .ninfer artifact of huihui‑ai/Huihui‑Qwen3.8‑27B‑abliterated (uncensored),
so you can run it on a single RTX 5090 without doing the quantize/convert yourself.
- Full native context: 262,144 tokens
- 4‑bit KV cache (K8V4) + optional host‑RAM KV offload
- DFlash2 speculative decoding (built into the artifact)
- ~120–180 tok/s decode on one 5090, ~3,470 tok/s prefill on long prompts
- OpenAI‑ and Anthropic‑compatible server
Authorized use only. Uncensored/abliterated weights remove the model's refusals.
Use this for red‑teaming your own apps, security research and offline experiments.
You are responsible for what you generate.
Requirements
- NVIDIA RTX 5090 (Blackwell, sm_120), 32 GB
- Ubuntu 24.04, CUDA 13.3
- NInfer built from source — github.com/Neroued/ninfer
Run
# 1) build NInfer (see its README), then:
export LD_LIBRARY_PATH=/usr/local/cuda-13.3/lib64:$LD_LIBRARY_PATH
hf download pcmaker/Huihui-Qwen3.8-27B-Uncensored-NInfer --local-dir ./art
./ninfer/build/apps/ninfer-serve ./art/huihui_uncensored_nvfp4.ninfer \
--host 0.0.0.0 --port 8188 \
--max-context 262144 \
--kv-dtype k8v4 \
--spec dflash2 --draft-tokens 3 \
--default-thinking-budget 600
Then call the OpenAI‑compatible endpoint at http://localhost:8188/v1/chat/completions.
Disable "thinking" per request with {"reasoning_effort": "none"}.
How it was built
Full step‑by‑step guide (build → quantize → convert → serve), scripts and benchmarks:
pcmaker/Qwen3.8‑27B‑Uncensored‑NInfer‑RTX5090‑Guide
In short: the uncensored BF16 base was quantized to the mixed FP8+NVFP4 layout that
NInfer's converter expects, then combined with unsloth/Qwen3.8-27B-NVFP4 (reference
layout) and z-lab/Qwen3.8-27B-DFlash2 (draft head) into this single .ninfer file.
Credits
huihui‑ai (uncensored base), the Qwen team (base model + tokenizer), unsloth (NVFP4
reference), z‑lab (DFlash2), and NInfer (engine).
All weights and tools belong to their respective authors and licenses.