license: apache-2.0
base_model:
- JonathanColetti/Qwen3.8-27B-Uncensored
base_model_relation: quantized
tags: - openvino
- int4
- awq
- qwen3.8
- qwen3_5
- uncensored
- abliterated
- vision
- mtp
- intel-arc
pipeline_tag: image-text-to-text
OpenVINO INT4 AWQ — asymmetric 4-bit weight compression, verified group size 128.
Qwen3.8 27B Uncensored — OpenVINO INT4 AWQ g128
OpenVINO INT4 AWQ deployment conversion of JonathanColetti/Qwen3.8-27B-Uncensored, based on Qwen/Qwen3.8-27B.
Jonathan Coletti performed the original refusal-direction removal and published the BF16 source checkpoint. Qwen created the base model. Wondernutts performed this OpenVINO conversion and controlled deployment validation; this repository does not claim authorship of the upstream model work.
The original plain-INT4 OpenVINO baseline remains untouched at Wondernutts/Qwen3.8-27B-Uncensored-int4-ov.
Verified conversion facts
The uploaded openvino_config.json and conversion manifest record:
- Weight format: asymmetric INT4
- Compression method: AWQ
- Group size: 128
- Dynamic-quantization group size: 128
- Compression ratio: 1.0
- OpenVINO/Optimum export metadata: Optimum 2.3.0, Transformers 5.2.0
- Source revision:
JonathanColetti/Qwen3.8-27B-Uncensored@5bb7aa90f0efef548e87005b1fb7658e522b6b7f - Base revision:
Qwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 - Source tensor inventory: 1,199 keys, including all 15 verified
mtp.*source tensors - Complete multimodal OpenVINO deployment artifacts: language model, tokenizer/detokenizer, text embeddings, and vision embedding/merger/position graphs
MTP caveat
The BF16 source checkpoint contains all 15 verified MTP tensors, restored from the Qwen base model and checked before conversion. However, the standard OpenVINO VLM export used here does not expose a separate MTP draft head. Do not interpret the source-tensor verification as a claim that this OpenVINO package performs MTP speculative decoding.
Controlled A/B validation
The repository includes AB_VALIDATED.json, comparing this AWQ candidate against the untouched plain-INT4 Wondernutts baseline. The record reports passed: true; both models produced the expected math answer, coherent roleplay output, and correct red-image recognition.
| CPU DQ0 smoke test | AWQ g128 candidate | Plain INT4 baseline |
|---|---|---|
| Load | 11.78 s | 11.27 s |
| Math generation | 38.67 s | 38.73 s |
| Roleplay generation | 117.26 s | 117.34 s |
| Vision generation | 66.69 s | 66.91 s |
This controlled CPU DYNAMIC_QUANTIZATION_GROUP_SIZE=0 smoke A/B shows no meaningful generation-latency regression in these three checks. It is a functional comparison, not a throughput benchmark: no tokens-per-second claim is made, and the small timing differences should not be generalized beyond this run and configuration.
The validation file also records that the candidate is AWQ while the preserved baseline is not; both use group-128 INT4 storage.
Source-model context
The source model substantially reduces—but does not eliminate—refusal behavior from Qwen3.8-27B using Heretic. Its source card reports the original methodology, measured behavior, limitations, and safety caveats. Review both the uncensored source card and Qwen base card before deployment.
Wondernutts OpenVINO work
- Gemma/Qwen conversion, validation, and serving tooling: Wondernuttz/OpenVino-For-Gemma-4
- Custom OpenVINO runtime research: Wondernuttz/openvino
These links provide conversion/runtime provenance. They are not claims of authorship over the Qwen or JonathanColetti source weights.
Inference with OpenVINO GenAI
import openvino_genai as ov_genai
from huggingface_hub import snapshot_download
model_dir = snapshot_download(
"Wondernutts/Qwen3.8-27B-Uncensored-int4-awq-g128-ov"
)
pipe = ov_genai.VLMPipeline(
model_dir,
"GPU",
DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)
config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 1.0
config.top_p = 0.95
config.top_k = 20
result = pipe.generate("Write a short scene in a candlelit inn.", generation_config=config)
print(result)
For direct answers, apply the repository's chat template with thinking disabled where supported. Native image input requires the multimodal VLMPipeline API and a compatible OpenVINO GenAI build.
Notes
- This is an OpenVINO deployment repository, not a Transformers checkpoint. Use OpenVINO/OpenVINO GenAI rather than
AutoModelForImageTextToText. - Reduced-refusal behavior is not the same as unrestricted correctness. The source model retains documented refusal and capability limitations.
- License: Apache-2.0, inherited from the source/base model. Review upstream terms before redistribution or commercial deployment.