license: apache-2.0
base_model:
- huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
language: - en
- zh
tags: - gguf
- abliterated
- uncensored
- imatrix
- convrot
pipeline_tag: image-text-to-text
Huihui-Qwen3.8-27B-abliterated — UD-Q6_K_XL-CR-imatrix
An experimental ConvRot GGUF quantisation of huihui-ai/Huihui-Qwen3.8-27B-abliterated, based on the tensor types of Unsloth's Dynamic V3.0 UD-Q6_K_XL with Q5_K / Q6_K / Q8_K tensors swapped to Q5_CR / Q6_CR / Q8_CR respectively.
Fidelity benchmark
The quantised model was compared against the BF16 model. Two separate held-out evaluation corpora were used:
- English: ~64k tokens
- Chinese: ~64k tokens
English summary
| Metric | Result |
|---|---|
| BF16 PPL | 6.008330 |
| Quant PPL | 6.011147 |
| PPL ratio | 1.000469 |
| Relative PPL increase | +0.0469% |
| Mean KLD | 0.000508 |
| Median KLD | 0.000195 |
| p95 KLD | 0.001682 |
| p99 KLD | 0.005023 |
| p99.9 KLD | 0.020512 |
| RMS Δp | 0.652% |
| Same top prediction | 98.965% |
Chinese summary
| Metric | Result |
|---|---|
| BF16 PPL | 10.854175 |
| Quant PPL | 10.860803 |
| PPL ratio | 1.000611 |
| Relative PPL increase | +0.0611% |
| Mean KLD | 0.000725 |
| Median KLD | 0.000331 |
| p95 KLD | 0.002420 |
| p99 KLD | 0.007094 |
| p99.9 KLD | 0.023688 |
| RMS Δp | 0.710% |
| Same top prediction | 98.721% |
Vision module
The model supports an optional vision module for visual inputs, but this repo does not include the corresponding mmproj GGUF.
Please refer to unsloth/Qwen3.8-27B-GGUF to download the mmproj GGUF.
Importance matrix
The quantisation uses an independently collected imatrix mainly composed of Chinese & English.
See IMATRIX.md for more information.
Quantisation method
The quantisation method is a proof of concept which covers ConvRot tensors and token embedding and MTP layers.
See QUANTIZATION.md for more information.
How to use it
This model contains non-official CR tensor types. You therefore need a llama.cpp-compatible runtime that implements the corresponding:
Q5_CR
Q6_CR
Q8_CR
tensor formats and kernels, such as TheTom/llama-cpp-turboquant.