license: apache-2.0
license_link: LICENSE
base_model: huihui-ai/Huihui-Qwen3.5-4B-abliterated
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
- qwen
- qwen3.5
- gguf
- llama.cpp
- abliterated
- uncensored
Huihui-Qwen3.5-4B-abliterated-GGUF
This repo holds one GGUF file of huihui-ai/Huihui-Qwen3.5-4B-abliterated. That model is an abliterated (refusal-removed) build of Qwen/Qwen3.5-4B.
The file is a byte-for-byte copy of mradermacher's imatrix quant in mradermacher/Huihui-Qwen3.5-4B-abliterated-i1-GGUF. It has not been requantized or modified. It is republished here, under the Apache-2.0 license, as the Copper tier of DeepSeek Unchained. All credit goes to the Qwen team for the model, to huihui-ai for the abliteration, and to mradermacher for the quantization.
Disclaimer
This model has had its safety alignment substantially removed. It will comply with requests that the original Qwen3.5-4B refuses. You are responsible for how you use it and for what it generates. Use it within the Apache-2.0 license and applicable law. Abliteration removes refusals; it does not add knowledge or remove biases from the base model's training. huihui-ai's model card has the full disclaimer.
Files
| Path | Quant | Size (bytes) | sha256 |
|---|---|---|---|
Huihui-Qwen3.5-4B-abliterated.i1-Q4_K_M.gguf |
i1-Q4_K_M (imatrix) | 2,707,514,944 | 7271e08993f5dbaf9c8fcdaf727c0fb2e42c21beea0d1f27e2a943bed4e81b64 |
The hash matches the file in the mradermacher repo. This is the text-only file, with no mmproj. It needs a recent llama.cpp with the qwen35 architecture, and it was tested with b10919. Qwen3.5 thinks by default; pass enable_thinking=false through the chat template (llama.cpp: --reasoning off) to turn it off.
How the quant was chosen
Eight quants from mradermacher's static and imatrix (i1-) repos were measured against the F16 file from the static repo, on one Apple M4 Max with 128 GB of memory. KL divergence against the F16 logits comes from llama.cpp b10919 llama-perplexity (wikitext-2-raw test, ctx 512, 100 chunks). Speed comes from llama-bench.
| Quant | File GB | Mean KLD | p99 KLD | Same top token % | PPL ratio vs F16 | pp512 t/s | tg128 t/s |
|---|---|---|---|---|---|---|---|
| i1-IQ3_M | 2.16 | 0.1635 | 1.504 | 83.00 | 1.0564 | 1051.5 | 64.08 |
| Q3_K_M | 2.26 | 0.1971 | 1.882 | 81.49 | 1.0943 | 1045.2 | 61.43 |
| IQ4_XS | 2.45 | 0.0781 | 0.786 | 88.58 | 1.0267 | 1135.5 | 67.79 |
| Q4_K_M | 2.71 | 0.0681 | 0.650 | 89.39 | 1.0435 | 1141.5 | 71.33 |
| i1-Q4_K_M | 2.71 | 0.0435 | 0.405 | 91.32 | 1.0363 | 1145.2 | 69.98 |
| Q5_K_M | 3.11 | 0.0194 | 0.186 | 94.23 | 1.0006 | 1154.2 | 62.82 |
| Q6_K | 3.46 | 0.0068 | 0.059 | 96.49 | 1.0053 | 1229.9 | 61.82 |
| Q8_0 | 4.48 | 0.0007 | 0.005 | 98.91 | 1.0050 | 1280.3 | 55.29 |
The imatrix i1-Q4_K_M is the same size and speed as the static Q4_K_M, and its mean KLD is about a third lower, so it was chosen. Lower KLD means the quant's output distribution is closer to F16. "Same top token" is how often the quant and F16 agree on the most likely next token.
License
Apache-2.0, inherited from Qwen/Qwen3.5-4B. The LICENSE file is copied from that repo.