library_name: openvino
license: apache-2.0
base_model:
- huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated
pipeline_tag: image-text-to-text
tags: - openvino
- int8
- qwen2.5-vl
- vlm
- abliterated
- uncensored
- intel
- arc
Qwen2.5-VL-7B-Instruct-abliterated — OpenVINO IR (int8)
OpenVINO IR conversion of
huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated,
an abliterated (decensored) build of Qwen2.5-VL-7B-Instruct.
This is a full VLM export: vision tower, vision-language merger, text embeddings,
language model, tokenizer and detokenizer are all present.
Sibling build: int4
Details
| Property | Value |
|---|---|
| Architecture | Qwen2_5_VLForConditionalGeneration, 28 text layers, hidden 3584 |
| Format | OpenVINO IR, int8 asymmetric per-channel |
| Repository size | 9.3 GB |
| Export type | decomposed VLM (separate vision / merger / text-embedding graphs) |
Because the text graph has more than the two inputs LLMPipeline expects, load this
through VLMPipeline, or serve it with OVMS which dispatches automatically.
Benchmark
Intel Core Ultra 7 258V (Arc 130V/140V iGPU), 30 GB RAM, OpenVINO Model Server
2026.4.0 on GPU, greedy decoding, 128 max new tokens, mean of 3 runs.
| Build | tok/s |
|---|---|
| int4 | 22.6 |
| int8 (this build) | 13.0 |
Decode here is memory-bandwidth bound, so throughput tracks model size closely.
Serving
cat > ovms_config.json <<'EOF'
{
"model_config_list": [
{
"config": {
"name": "qwen2.5-vl",
"base_path": "/absolute/path/to/this/model",
"target_device": "GPU",
"nireq": 8,
"plugin_config": { "PERFORMANCE_HINT": "THROUGHPUT", "NUM_STREAMS": "2" }
}
}
]
}
EOF
# graph.pbtxt must also exist in the model directory.
PYTHONPATH=$OVMS_ROOT/lib/python ovms --config_path ./ovms_config.json --rest_port 11436
Conversion notes
- Stage 1:
optimum-cli export openvino --task image-text-to-text --weight-format fp16 - Stage 2:
nncf.compress_weightson the language-model IR. - Requires transformers 5.0 — the
qwen2_5_vlexporter rejects newer versions
(MAX_TRANSFORMERS_VERSION = "5.0"). Newer transformers fails with a version-gate error. - optimum-intel 2.2.0 / OpenVINO 2026.4.0.
Limitations
- Abliterated: reduced refusal behavior. Evaluate outputs before deployment.
- Quantized; re-export at a smaller group size or fp16 for maximum accuracy.
- Throughput depends on runtime kernels, hardware, and prompt distribution.
Attribution
Base model: huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated.
Qwen2.5-VL is Apache-2.0. Conversion via optimum-intel + NNCF.