license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- gguf
- llama.cpp
- qwen3_5
- abliterated
- uncensored
- mtp
Huihui Qwen3.8 27B Coder390 EfficientThink abliterated: Q4_K_M GGUF
A Q4_K_M quantization of
huihui-ai/Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated
that keeps the model's MTP layers, so llama.cpp can use them for self-speculative drafting. We made it on 2026-10-11, before huihui-ai's own GGUF repo existed, and it has been our daily 27B model since 2026-10-12. That repo has no Q4_K_M. mradermacher publishes Q4_K_M builds of the same model, which are different files from this one.
The upstream model is abliterated: its refusal behaviour has been removed, so it has no built-in guardrails.
File
| File | Size | SHA256 |
|---|---|---|
Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-Q4_K_M.gguf |
16,810,715,136 bytes | 299b3c3d0e8713a99b0c926ebb85d79370fce27cee166decc3e3a7217c06fed6 |
The file has 866 tensors and architecture qwen35. Its metadata records no imatrix, so treat it as a plain Q4_K_M.
Running it
llama-server \
-m Huihui-Qwen3.8-27B-Coder390-EfficientThink-abliterated-Q4_K_M.gguf \
--mmproj <vision projector> \
-ngl 999 -c 65536 -np 1 \
--spec-type draft-mtp --spec-draft-n-max 2
We run this on llama.cpp master f74e2fa (2026-10-16). Older builds may not have --spec-type draft-mtp.
Vision projector: this repo doesn't include one, because our original is gone and we haven't rebuilt it. We use mradermacher's .mmproj-f16.gguf from their repo. It read a simple test image correctly with this model.
What we measured
On one RTX 3090 (24 GB), with the flags above and the vision projector on CPU. Speed depends heavily on how many drafts are accepted, which depends on the text. These are medians from our server logs over four weeks of daily use, on requests of 100 tokens or more:
| Draft acceptance | Generation speed | Requests |
|---|---|---|
| 40-55% | 42.5 tokens/s | 70 |
| 80-90% | 56.6 tokens/s | 3,158 |
| 90-100% | 62.9 tokens/s | 6,930 |
Long runs go faster: requests of 2,000 tokens or more had a median of 74 tokens/s at 86% acceptance (154 requests), and the fastest run of 500 tokens or more reached 89 tokens/s. In a short test, prose accepted about 50% of drafts (42-44 tokens/s) and code about 85% (55-56 tokens/s). These logs come from a machine doing other work, so they aren't controlled benchmarks.
What we did not measure
This build has no quality metrics. We set up a comparison against the BF16 weights (KL divergence, top-1 agreement, code tests), but it never finished for this file, and an independent review of that method stayed at "pending". We can't give you KL or top-1 numbers. Our evidence is four weeks of daily use, which isn't a benchmark.
License
Apache-2.0, following the upstream model. See LICENSE. Credit for the model belongs to huihui-ai and the upstream authors listed on their card, including nerkyor.