library_name: gguf
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
tags:
- gguf
- qwen3
- uncensored
- mtp
- speculative-decoding
- 16gb
Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored
Uncensored 27B dense hybrid (48 Gated-DeltaNet + 16 full-attention layers, 64 total + 1 MTP head) quantized with non-uniform GSQ-RCO per-tensor allocation on the Huihui abliterated base. Built for single-GPU 16GB agentic work: fully GPU-resident, native MTP speculative decoding, 196K context.
- Base:
huihui-ai/Huihui-Qwen3.8-27B-abliterated(refusal-ablated via orthogonal projection; MTP head intact) - Method: GSQ (Gumbel-Softmax, arXiv:2604.18556) + RCO (Riemannian Constrained Optimization, arXiv:2605.00649), ISTA-DASLab toolchain
- Importance: official
imatrix-qwen3.8-27b.gguf(1000×4096 tok); custom syncretized blends tried (v2/v3) and reverted after needle regressions - File: 10,442,827,584 B (9.73 GiB), 866 tensors, ~3.06 bpw,
blk.64.nextn.*MTP head preserved (q6_K) - Template: Froggeric v22.5 + terse system prompt,
tool_call_format=json,reasoning-effort medium, budget 256
Measured (RTX 5070 Ti 16GB, build2 llama.cpp, greedy unless noted)
| Config | Decode | Notes |
|---|---|---|
| no-spec baseline | 39.6 t/s | 400 tok prose, thinking on |
MTP --spec-draft-n-max 2 |
59.3 t/s (+50%) | ship default; draft accept 0.81-0.95, mean len 2.6-2.9 |
| MTP n-max 3 + p-min 0.7 | 33.4 t/s | verification cost dominates — do not use |
| MTP n-max 4 + p-min 0.75 | 36.5 t/s | 84% accept yet slower — accept ≠ speed |
KV iq4_nl |
26.8 t/s | no fast kernel in build2 — do not use |
KV q4_0/q4_0 |
ship | 15.8/16GB resident at 196K |
- Needle 6/6 at 196K (
q4_0/q4_0);q8_0/q8_0recovers nothing extra and spills (16.5GB) — 150K max for q8 KV. - Toolcall: 8/8 v1 (Pi json path); v2 raw-template 7/14 (negatives 4/4 clean; schema-path hits build2 tools+MTP post-boot flake — Pi-path number pending); WRONG_ARGS + chains scored explicitly.
- Temp: 0.6 ships (4/8 vs 3/8 raw-template A/B; determinism for agentic/medical use).
- Reasoning: medium default; low for short code (medium overthinks it); xhigh for verifiable-correctness only (9/12 vs 6-7/12 SWE-style per community).
Honest limits
- MTP output is deterministic per config but not byte-identical to serial greedy on quantized targets (community: 92-100% divergence at 1600 tok). Quality-neutral on our gates; do not use where byte-parity is contractual.
- Knowledge recall regressed vs Qwen3.6 (agentic tradeoff — it searches instead of recalling). Pair with RAG/MCP or a knowledge sidecar.
- Vision + 64K + MTP leaves ~150MB on 16GB — split text/vision profiles or
--no-mmproj-offload. n_rs_seq = draft.n_max: deeper drafts cost recurrent-snapshot VRAM — the mathematical reason n-max 2 is optimal on 16GB.
Run it
llama-server -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS-Uncensored-MTP.gguf `
--alias XS-Pro-27B --ctx-size 196608 -fa on -ctk q4_0 -ctv q4_0 `
-ngl 99 -b 512 -ub 512 -np 1 -t 8 --jinja `
--reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2
Versions
- v1 (ship): official imatrix, REF allocation, MTP head intact. 29/30 suite.
- v2/v3 (reverted): fresh/syncretized imatrix — 0/6 needle timeouts. Archived.
- v1.1 (planned): FastMTP-style head FT on 4M self-distilled diverse corpus → MTP3 depth. This card updates on release.