license: apache-2.0
base_model: llmfan46/gemma-4-12B-uncensored-heretic
tags:

• openvino
• int4
• gemma-4
• heretic
• uncensored
• vision-language-model
• text-generation
• image-text-to-text

Gemma-4 12B Fable5 — OpenVINO INT4 (Uncensored)

This is an OpenVINO INT4 conversion of gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2, an uncensored variant of Google's google/gemma-4-12B-it.

Conversion

CLI command

optimum-cli export openvino \
    --model llmfan46/gemma-4-12B-uncensored-heretic \
    --task image-text-to-text \
    --weight-format int4 \
    --trust-remote-code \
    gemma-4-12B-heretic-int4-ov

Model size

Hardware requirements

• GPU VRAM: 8 GB minimum (tested on Arc A770 16 GB)
• RAM: 16 GB+
• Recommended context: 8K tokens at ~10 tok/s on Arc A770

Usage

With OpenVINO GenAI (Python)

import openvino_genai as g

pipe = g.VLMPipeline(
    "gemma-4-12B-heretic-int4-ov",
    "GPU",
    DYNAMIC_QUANTIZATION_GROUP_SIZE=0,
    GPU_ENABLE_LARGE_ALLOCATIONS=True,
)

# Raw text generation
result = pipe.generate("The capital of France is", max_new_tokens=10)
print(result.texts)

# Chat mode
pipe.start_chat()
result = pipe.generate("Hello!", max_new_tokens=50)
print(result.texts)
pipe.finish_chat()

With OpenAI-compatible server

python ov_server.py
# or
OV_MODEL=../gemma-4-12B-heretic-int4-ov \
GPU_ENABLE_LARGE_ALLOCATIONS=True \
DYNAMIC_QUANTIZATION_GROUP_SIZE=0 \
python ov_server.py

curl http://localhost:8092/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Hello!"}]}'

Multimodal (image understanding)

from PIL import Image
import numpy as np
import openvino as ov

pipe = g.VLMPipeline("gemma-4-12B-heretic-int4-ov", "GPU")
image = Image.open("photo.jpg")
img_tensor = ov.Tensor(np.array(image.convert("RGB"))[None])

result = pipe.generate("Describe this image", image=img_tensor, max_new_tokens=200)
print(result.texts)

Performance (Arc A770 16GB)

License

Apache 2.0 (same as the source model)