license: apache-2.0
license_link: LICENSE
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• qwen3.8
• abliterated
• uncensored
• w4a16
• 4-bit
• int8
• compressed-tensors
• autoround
• vllm
• marlin
• mtp
• speculative-decoding
• reasoning
• function-calling
• vision-language
• rtx-3090

Orca-Qwen3.8-27B-Uncensored-W4A16

A 4-bit (W4A16) quantization of
orcarouter/Qwen3.8-27B-Uncensored,
the abliterated (refusal-removed) BF16 build of Qwen/Qwen3.8-27B.
It serves on one 24 GB GPU (RTX 3090 class) with vLLM and the
HyperQwen stack, with the vision tower, the MTP
speculative-decoding head and tool calling kept.

A faster variant with int4 heads is
Orca-Qwen3.8-27B-Uncensored-W4A16-fast.

Lineage

Qwen/Qwen3.8-27B                         Apache 2.0
 └─ orcarouter/Qwen3.8-27B-Uncensored    BF16, refusal direction removed (abliteration)
     └─ this model                       W4A16 g128 body, int8 lm_head / embed_tokens / MTP
         └─ ...-W4A16-fast               int4 GPTQ lm_head / MTP, own-output draft vocabulary

What is in this checkpoint

2,022 tensors, 67 safetensors files, 16.7 GB on disk. No group has zero points.

The draft vocabulary is the generic 40,960-id list that the official
Qwen3.8-27B-W4A16-AutoRound checkpoint also uses. On this model's own outputs it
covers 98.39 % of held-out tokens; a list counted over the model's own outputs covers
97.90 % (see the fast variant for how that was measured).

How it was made

1. run_quant.sh from HyperQwen: Intel AutoRound, --scheme W4A16 --bits 4 --group_size 128, 128 calibration samples x 2,048 tokens, lm_head not quantized,
   auto_round:llm_compressor export.
2. prepare/quant_lm_head.py, prepare/quant_embed.py, prepare/quant_mtp.py: int8
   round-to-nearest heads. Relative error against the BF16 source weights
   (||dequant - w|| / ||w||): lm_head 0.0064, MTP linears 0.0066–0.0153.
3. prepare/build_draft_vocab.py --ids prepare/draft_vocab_ids.json: the draft head.

The abliteration itself is a property of the BF16 source and is carried through
unchanged. orcarouter reports (on their own block-FP8 build, rule-based classifier):
AdvBench refusal 0.0 % vs 99.0 % for the base model, MMLU 84.7 % vs 84.3 %, GSM8K CoT
88.7 % vs 90.0 %. See the source card
for the method and the full tables.

Measured results (single RTX 3090, 24 GB)

All numbers use the HyperQwen stack on vLLM 0.28.0, WSL2 + Docker, thinking off
(protocol v2). The official Qwen3.8-27B-W4A16-AutoRound checkpoint is the reference.

Single-user decode (bench/run_benchmarks.sh single), 2026-09-23, card at 250 W,
SPEC=dflash2 (DFlash2 drafter, 7 draft tokens), PREFIX_CACHE=1, KV_MEM=4529848320
(4.22 GiB), MAX_LEN=49152. End-to-end throughput in tok/s, sampling T=default / T=0.
One run per checkpoint after a warm-up request.

C1 accepts 3.71 / 3.97 tokens per verify step, mean time to first token 186 ms. Expect
3–5 % variation between sessions.

Batch serving (bench/run_benchmarks.sh batch --prefill --long), 2026-09-21, card at
350 W, KV=fp8, no speculation, second of two runs.

Quality (bench/quality_battery.py against the served model): perplexity over
~300-token windows of wikitext-2 (en), fineweb-2 Danish (da) and Python source (code);
GSM8K exact match, first 200 test questions, greedy, thinking off.

With n=200 the GSM8K standard error is about 1.6 points.

How to serve

HyperQwen (Linux or WSL2, one 24 GB GPU), in .env:

MODEL=/app/models/Orca-Qwen3.8-27B-Uncensored-W4A16
SPEC=dflash2          # or SPEC=mtp to draft with the built-in MTP head
PREFIX_CACHE=1

then docker compose --profile single up -d. bash verify.sh --no-server checks the
directory before you serve it.

Plain vLLM (0.28 or later) loads the body and heads through compressed-tensors:

vllm serve TyroneNel/Orca-Qwen3.8-27B-Uncensored-W4A16 \
  --max-model-len 32768 --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder

The truncated MTP draft head (mtp.draft_lm_head.* with mtp_draft_vocab_ids.pt) is
read by HyperQwen's patches/qwen3_5-mtp-draft-vocab.patch. It is not tested on
unpatched vLLM.

Sampling defaults (generation_config.json): temperature 1.0, top_p 0.95, top_k 20.
The chat template accepts reasoning_effort low, medium and xhigh (default), and
maps minimal to low and high / max to xhigh.

Files

Disclaimer

This model has its safety alignment removed. It will follow harmful, unethical or
illegal requests that the original model refuses. It is for research: refusal
mechanisms, interpretability, red-teaming and robustness evaluation. Do not put it in
front of end users without your own moderation layer. You are responsible for how you
use it and for what it generates. Its outputs do not represent the views of the
uploader, OrcaRouter, or Qwen / Alibaba.

License

Apache License 2.0, inherited from Qwen/Qwen3.8-27B
through the OrcaRouter release. Changes made in this work: the weights are quantized as
described above, the chat template is extended, and this README is replaced.