tags:
- 27B
- MTP
- GGUF
- Vision
- Ollama
- llama.cpp
- text
- Uncensored
- NVFP4
- QWEN3.8
base_model: - Qwen/Qwen3.8-27B
- HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF
Request a model Quant in GGUF - Contact : [email protected]
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive — NVFP4 Mixed Re-Quant
A tensor-level NVFP4 re-quantization of the HauhauCS Qwen3.8-27B-Uncensored-HauhauCS-Aggressive model, built for NVIDIA Blackwell (RTX 50-series) local inference.
This release takes the HauhauCS Q8_K_P text model (31.46 GB, 9.21 bpw) and re-quantizes it to a mixed NVFP4 / Q8_0 / F32 / F16 profile (17.10 GB, 5.00 bpw) — cutting VRAM by ~15 GB while preserving the model's native MTP/NextN head, vision, tokenizer, and quality-critical tensors. It runs on standard GGUF runtimes and leverages Blackwell's native FP4 tensor cores for the quantized compute matrices.
Credit: Based on the Qwen and HauhauCS teams' work. This is a re-quantization of the HauhauCS Aggressive uncensored variant — no changes to datasets, weights' values, or intended capabilities.
What is this?
The HauhauCS Aggressive release is a 27B dense causal language model with a vision encoder: direct answers, no refusal behavior, minimal preamble on hard prompts (0/465 refusals). It preserves Qwen3.8's native NextN (MTP) head and offers the HauhauCS FastMTP acceleration sidecar.
This NVFP4 mixed re-quant keeps all of that intact and makes the model fit comfortably on a 24 GB Blackwell card (RTX 5060 Ti / 5070 / 5080) with room for a large context, while keeping the quality-sensitive tensors at high precision.
Why NVFP4?
NVFP4 (4-bit floating point, 16-element sub-block scaling) is the native precision of Blackwell's FP4 tensor cores. Re-quantizing the large compute-heavy matrices to NVFP4:
- Halves VRAM vs. the Q8_K_P source (31.46 GB → 17.10 GB).
- Uses the GPU's fastest tensor cores for the dominant matmuls (FFN, attention projections, DeltaNet output).
- Stays near Q5-class quality because the quality anchors (output head, embeddings, recurrent state, norms, MTP head) are kept at Q8_0 / F32 / F16.
The re-quant was done from the Q8_K_P GGUF (9.21 bpw — high enough to produce a clean NVFP4 result) using an importance matrix, so no BF16 source was required.
Files
| File | Description | Size | BPW |
|---|---|---|---|
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-mixed.gguf |
This release — mixed NVFP4 re-quant of Q8_K_P | 17.10 GB | 5.00 |
mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf |
Vision projector (unchanged, from HauhauCS) | 931 MB | — |
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf |
HauhauCS FastMTP 32K acceleration sidecar (unchanged) | 903 MB | — |
The projector and FastMTP sidecar work with this NVFP4 model exactly as they do with any HauhauCS text quant. Download the projector only if you need image/video input; the FastMTP sidecar is optional and requires the HauhauCS runtime patch (see below).
The quantization recipe (tensor-level)
This is the core of the release — a per-tensor-type mixed profile. 866 tensors total, verified after quantization:
| Type | Count | Tensors | Rationale |
|---|---|---|---|
| NVFP4 | 371 | ffn_gate / ffn_up / ffn_down (64 layers); DeltaNet attn_qkv / attn_gate / ssm_out (48); full-attention attn_q / attn_v (16) |
The large compute-heavy matrices — biggest VRAM + Blackwell speed win |
| Q8_0 | 115 | token_embd, output (lm_head), ssm_beta / ssm_alpha / ssm_convNd (48 DeltaNet recurrent-state), blk.64.nextn.eh_proj (MTP), all attn_output |
Quality anchors + recurrent state (error-compounding) + MTP draft path |
| F32 | 360 | all norms (attn_norm, attn_post_norm, attn_q_norm, attn_k_norm, ssm_norm, nextn.enorm / hnorm / shared_head_norm) + ssm_a / ssm_dt |
Tiny, error-sensitive — kept exact at negligible cost |
| F16 | 16 | MTP layer-64 attn_q / attn_k / attn_v / attn_output |
The draft head's attention kept exact to protect FastMTP acceptance |
Design principles
- NVFP4 every large compute matrix — that's where VRAM and Blackwell speed live.
- Keep quality-critical and delicate tensors high-precision — the output head (lm_head) and token embeddings anchor output quality; the DeltaNet recurrent state (
ssm_beta/alpha/conv1d) compounds error across the sequence; norms are tiny and must stay exact. - Protect the MTP head — the MTP/NextN draft head is tied to
token_embd(kept Q8_0), andnextn.eh_projis kept Q8_0, so the FastMTP draft acceptance rate (and its 3× speed) is preserved. attn_outputkept Q8_0 — attention output projections stay high-precision for quality.
Method
- Importance matrix generated with
llama-imatrixon the Q8_K_P source (45 calibration chunks spanning code, prose, reasoning, JSON, and agentic prompts). - Re-quantization with
llama-quantize --allow-requantize --imatrix <imatrix> --tensor-type-file <recipe> --token-embedding-type q8_0 --output-tensor-type q8_0, from the Q8_K_P GGUF. - Verification — all 866 tensor types confirmed against the recipe after quantization.
How to run
Ollama
Import with the dual Modelfile (model + projector), then run:
FROM /path/to/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-mixed.gguf
FROM /path/to/mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 32768
PARAMETER num_predict 8192
PARAMETER repeat_penalty 1.0