library_name: openvino
base_model:
- HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive
- GitMylo/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-safetensors
tags: - openvino
- intel-npu
- int4
- qwen3.5
- uncensored
- optimum
pipeline_tag: image-text-to-text
license: apache-2.0
Qwen3.5-9B-Uncensored-HauhauCS-Aggressive — OpenVINO INT4 (Intel NPU Optimized)
This repository contains the OpenVINO™ Intermediate Representation (IR) model for Qwen3.5-9B-Uncensored-HauhauCS-Aggressive, optimized for Intel NPU (Neural Processing Unit, such as Intel® Core™ Ultra processors) as well as Intel CPU and GPU.
The model weights are compressed to INT4 Symmetric format using NNCF (Neural Network Compression Framework) via Optimum Intel, meeting Intel NPU hardware execution specifications.
Model Highlights
- Base Model: HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive
- Safetensors source: GitMylo/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-safetensors
- Quantization: INT4 Symmetric (
sym=True,group_size=128,ratio=1.0) - Target Hardware: Intel NPU (AI Boost), Intel CPU, Intel Arc / Iris Xe GPU
Requirements
pip install --upgrade openvino openvino-genai optimum-intel transformers
Inference on Intel NPU
1. Using OpenVINO GenAI (Recommended for low-latency NPU inference)
import openvino_genai as ov_genai
# Load model onto Intel NPU
pipe = ov_genai.VLMPipeline("JaelTmls/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-OpenVINO-INT4", device="NPU")
# For text-only generation:
prompt = "Explain quantum computing in simple terms."
response = pipe.generate(prompt, max_new_tokens=256)
print(response.texts[0])
(Note: If NPU device is not available, you can replace "NPU" with "GPU" or "CPU").
2. Using Optimum Intel
from optimum.intel.openvino import OVModelForVisualCausalLM
from transformers import AutoProcessor
model_id = "JaelTmls/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-OpenVINO-INT4"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = OVModelForVisualCausalLM.from_pretrained(model_id, device="NPU")
# Run generation
inputs = processor("Hello!", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=100)
print(processor.decode(output[0], skip_special_tokens=True))
Quantization Details
- Method: NNCF Weight Compression
- Weight format: INT4 Symmetric
- Group size: 128
- Ratio: 1.0 (all linear layers quantized to 4-bit)