license: apache-2.0
base_model: HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
tags:

• awq
• w4a16
• compressed-tensors
• vllm
• mtp
• speculative-decoding
• uncensored
• qwen3_5
  language:
• en

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive — AWQ W4A16 (MTP preserved)

4-bit AWQ (W4A16, group 128, asymmetric) build of
HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF,
with the native NextN/MTP draft head kept intact for speculative decoding under vLLM.

19 GB on disk. 9.66 GiB of weights per GPU on a 2xRTX 3090 (TP=2), leaving room for a
560,900-token KV cache at fp8.

Provenance — read this first

HauhauCS released this model as GGUF only. There is no upstream BF16 safetensors
checkpoint to quantize from, so this build was produced by reconstructing one:

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf   (29.3 GiB, sha256 4e7735df4d1e...)
mmproj-...-Aggressive-BF16.gguf                          (vision tower, bf16)
        |
        +- dequantise + reconstruct -> BF16 HF safetensors (52 GB)
        |
        +- llm-compressor AWQ -------> W4A16 compressed-tensors (19 GB)   <- this repo

This is a quant of a Q8_K_P quant, not of the original weights. The ~8-bit source
error is baked in and compounds with the 4-bit step. If HauhauCS ever publishes BF16
safetensors, a build made from those should be preferred over this one.

The vision tower is the exception: the mmproj GGUF ships bf16, and the tower is excluded
from quantization, so it is bit-identical to upstream.

Why a reconstruction was needed

qwen3_5 is a hybrid architecture — 64 decoder layers, of which 48 are
Qwen3_5GatedDeltaNet linear-attention and 16 are full attention (layer N is
full-attention iff N % 4 == 3) — plus a 27-layer vision tower and an embedded NextN
head. No off-the-shelf GGUF-to-HF converter handles it, so the mapping was written from
scratch and proven total and bijective: 866 GGUF tensors <-> 866 non-vision HF
tensors, plus 334 mmproj -> 333 HF vision tensors.

Three llama.cpp storage conventions had to be inverted. Each one loads and runs happily
if you get it wrong, and produces fluent nonsense:

The value-head ordering is the one that hurts. It affects in_proj_qkv, in_proj_z,
out_proj and conv1d on all 48 linear-attention layers. With it wrong the model loads,
serves, and answers every prompt with degenerate repetition.

Dequantization is done in fp32, not bf16: bf16 eps at 1.0 is 0.0078, so a norm weight
of 0.9666 rounds to 0.96875 and subtracting 1.0 yields -0.03125 against a true -0.0334 —
a 6% error on every normalization weight in the model.

Verification

Reconstruction was gated on a numeric check against a different fine-tune of the same
base, so it validates layout and storage convention, not weight identity:

• Correlation sweep — every linear-attention tensor at +0.99998; A_log/dt_bias
  +0.97917, norm.weight +0.99219 (small tensors, at the Q8_0 noise floor). This is the
  check that catches permutations; the broken build scored 0.04–0.73 here.
• Norm-offset sweep — all 19 norm families at +/-0.00000, confirming no residual +1.

Correlation cannot prove the weights are right, only that they are in the right shape
and convention. The behavioural tests below are what establish the model works.

Quantization recipe

Left in bf16 (ignore): lm_head, the entire 27-layer vision tower, the in_proj_a /
in_proj_b delta-rule scalars on all 48 linear-attention layers, and the MTP head.

If you re-quantize this yourself: llm-compressor never sees
model-mtp.safetensors, so the ignore list it emits omits the MTP module entirely.
You must add re:.*mtp.* and re:mtp\..* to quantization_config.ignore by hand, or
vLLM builds the drafter as W4A16, fails to find weight_packed, silently skips every
MTP tensor and dies with KeyError: 'weight' in qwen3_5_mtp.py.

Validation

All measured on this build, 2xRTX 3090 (TP=2), vLLM nightly, fp8 KV cache, MTP n=3.

Speed

Single-user (c=1), thinking disabled, 600-token generations:

MTP acceptance rate 64.9%, mean accepted length 2.95 of 4 — the draft head is
genuinely doing work, which is where the margin over the comparison build comes from.

No standardized benchmarks (MMLU, GSM8K, etc.) were run. The numbers above are
functional and performance checks, not a quality evaluation. Treat quality as unmeasured.

Serving with vLLM

vllm serve /path/to/this/repo \
  --served-model-name qwen3.8-27b-hauhau-aggressive \
  --tensor-parallel-size 2 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 4 \
  --max-num-batched-tokens 4096 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --enable-prefix-caching --enable-chunked-prefill \
  --compilation-config '{"cudagraph_mode": "PIECEWISE"}' \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --default-chat-template-kwargs '{"enable_thinking": true, "reasoning_effort": "medium"}'

Notes from getting this stable on 2x3090:

• Pin reasoning_effort to medium. The Qwen3.8 chat template defaults to xhigh,
  which spends the entire budget thinking and returns an empty answer. Measured on this
  build over 4 hard prompts: at xhigh, 2 of 4 produced no answer at all (~5,400
  words of reasoning, then finish_reason=length); at medium, 0 of 4 failed, averaging
  ~529 words of thinking and ~954 words of answer. The flag merges per-key, so clients
  sending their own chat_template_kwargs are still covered.
• cudagraph_mode: PIECEWISE, not FULL — FULL replay segfaults with MTP
  (vllm#40756) at identical c=1 throughput.
• On 3090s without working P2P, set NCCL_P2P_DISABLE=1 and --disable-custom-all-reduce.

Files

HauhauCS FastMTP is not used here — that is a llama.cpp-only draft sidecar needing
his runtime patch. This repo carries the native embedded NextN head (blk.64 -> mtp.*),
which the reconstruction preserves and vLLM consumes directly.

Provenance / credit

• Fine-tune: HauhauCS — all model quality and the
  uncensored behaviour are theirs.
• Base architecture: Qwen team, qwen3_5.
• This repo contributes only the GGUF-to-HF reconstruction and the AWQ quantization.

As of 2026-08-18 no AWQ/W4A16 build of any HauhauCS Aggressive model appears on the Hub
(searched model ids across 90 derivative repos; GGUF, NVFP4 and MLX builds exist).

Limitations

• Quantized from Q8_K_P, not from original BF16 weights — see Provenance.
• Verification establishes correct tensor layout and working behaviour, not weight-level
  fidelity to the upstream fine-tune.
• No standardized quality benchmarks were run.
• This is an uncensored model. It will answer requests that safety-tuned models
  refuse. You are responsible for how you use it.