base_model: huihui-ai/Huihui-Qwen3.5-9B-abliterated
license: apache-2.0
library_name: transformers
quantized_by: nchapman
tags:
- gptq
- w4a16
- compressed-tensors
- vllm
Huihui-Qwen3.5-9B-abliterated-W4A16-GPTQ
INT4 (W4A16) GPTQ quantization of huihui-ai/Huihui-Qwen3.5-9B-abliterated — the same
model at roughly a quarter of the memory, built for vLLM.
- Base model: huihui-ai/Huihui-Qwen3.5-9B-abliterated
- Calibration data: 999 samples at the dataset's own
proportions from nvidia/Nemotron-Cascade-2-SFT-Data; the
exact rows are pinned in
warp-3090-quant-lab-frozen-data - Chat template: froggeric Qwen-Fixed-Chat-Templates (v22.5)
ships aschat_template.jinja— the artifact's only template - Default sampling (
generation_config.json): temperature 1.0, top_p 0.95, top_k 20 — the thinking-mode
defaults of the base model's card; send your own values to override - Recipe and tooling: warp-3090-model-lab
Vision tower, MTP head, linear-attention projections, and lm_head stay BF16.
from vllm import LLM
llm = LLM(model="nchapman/Huihui-Qwen3.5-9B-abliterated-W4A16-GPTQ")