library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-27B/blob/main/LICENSE
pipeline_tag: image-text-to-text
base_model:
- huihui-ai/Huihui-Qwen3.6-27B-abliterated
base_model_relation: quantized
tags: - qwen3.6
- qwen3_5
- awq
- autoawq
- int4
- 4-bit
- w4a16
- gemm
- quantized
- safetensors
- vllm
- v100
- vision
- image-text-to-text
- abliterated
- uncensored
Huihui-Qwen3.6-27B-abliterated-AWQ
AWQ W4A16 quantized version of huihui-ai/Huihui-Qwen3.6-27B-abliterated.
This repository is marked as a quantized derivative of the Huihui model via:
base_model:
- huihui-ai/Huihui-Qwen3.6-27B-abliterated
base_model_relation: quantized
Quantization
The model uses native AutoAWQ-style AWQ INT4 weights with FP16 activations:
{
"quant_method": "awq",
"bits": 4,
"group_size": 128,
"version": "gemm",
"zero_point": true
}
Additional modules intentionally left unquantized are recorded in config.json under quantization_config.modules_to_not_convert.
Tested Runtime
Validated locally with a modified 1Cat-vLLM build on 4 x Tesla V100-SXM2-32GB:
python -m vllm.entrypoints.openai.api_server \
--model alexxorm/Huihui-Qwen3.6-27B-abliterated-AWQ \
--quantization awq \
--dtype float16 \
--tensor-parallel-size 4 \
--kv-cache-dtype fp8_e5m2
The tested local server used SM70 AWQ kernels, FLASH_ATTN_V100, and FP8 KV cache. For contexts above the model config limit, vLLM requires VLLM_ALLOW_LONG_MAX_MODEL_LEN=1; use that override only after validating quality/stability for your workload.
Notes
This model inherits the safety/usage characteristics of the upstream abliterated model. The upstream authors describe it as an uncensored/abliterated variant of Qwen3.6-27B and warn that safety filtering is reduced. Review outputs before using in production or public-facing systems.
Base Model
- Quantized from: huihui-ai/Huihui-Qwen3.6-27B-abliterated
- Original base model referenced by upstream: Qwen/Qwen3.6-27B