Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated-AWQ

This repository provides an AWQ-quantized version of the original
huihui-ai/Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated model.

The goal of this quantization is to reduce GPU memory usage and enable efficient inference deployment using frameworks such as vLLM.

Model Details

• Base model: huihui-ai/Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated
• Quantization: AWQ
• Quantization tool: llm-compressor
• Precision: INT4 weight quantization
• Target inference engine: vLLM

Quantization Process

The model was quantized using llm-compressor with a calibration dataset and standard AWQ workflow.

Key steps:

1. Prepare calibration dataset
2. Run AWQ quantization using llm-compressor
3. Export quantized weights compatible with vLLM
4. Upload quantized model to HuggingFace

Intended Usage

This model is intended for:

• Efficient multimodal inference
• Deployment with vLLM
• Reduced VRAM consumption compared to FP16 models

Example use cases include:

• conversational agents
• multimodal assistants
• experimentation with large multimodal models on limited hardware

Acknowledgements

All credit for the original model goes to:

huihui-ai / Huihui-Qwen3-VL-30B-A3B-Instruct-abliterated

This repository only provides an AWQ-quantized version for inference efficiency.