license: gemma
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
library_name: transformers
base_model: google/gemma-4-12B-it
tags:
- gemma
- gemma4
- gemma-4
- unified
- abliterated
- uncensored
- multimodal
- vision
- audio
pipeline_tag: any-to-any
gemma-4-12B-it-abliterated-uncensored-exl3-4bpw
Overview
EXL3 4bpw quantized version of gemma-4-12B-it-abliterated-uncensored — an abliterated, uncensored variant of google/gemma-4-12B-it (Gemma 4 12B Unified, dense, ~11.95B parameters).
Important loading note
This repository contains an EXL3 4.0 bpw quantized checkpoint intended for EXL3-compatible runtimes such as ExLlamaV3 / TabbyAPI.
The Hugging Face model page may automatically display a generic Transformers loading snippet such as AutoProcessor / AutoModelForMultimodalLM. That snippet is not the recommended loading path for this EXL3 quantized checkpoint.
For normal use, load this model with an EXL3-compatible runtime and point the runtime at this repository or the downloaded model folder.
Metadata note
Hugging Face's automatic sidebar metadata may not accurately represent EXL3 quantized checkpoints. In particular, automatically displayed parameter counts or tensor-type summaries may be misleading for this format.
The source model is Gemma 4 12B dense from OpenYourMind/gemma-4-12B-it-abliterated-uncensored; this repository contains a 4.0 bpw EXL3 quantized export, not the original full-precision Transformers checkpoint.
Quantization
EXL3, 4.0 bits per weight (4bpw)
Intended runtime: ExLlamaV3 / TabbyAPI-compatible
Local Validation
- Exported and loaded locally on RTX 4090 24GB
- Clean-load VRAM observed around 7220 MiB
- Text generation smoke tested
enable_thinking=falsetested with valid JSON and no channel-marker leakageenable_thinking=truetested with valid JSON and correct answer- Image input not tested
- Audio input not tested
- This model does not provide TTS/audio output
Compatibility Notes
- This is not a full-precision Transformers checkpoint.
- Gemma 4 unified architecture may require runtime compatibility patches in ExLlamaV3/TabbyAPI.
- Output config was patched for ExLlamaV3/TabbyAPI compatibility.
chat_template.jinjawas aligned with validated Gemma 4 12B EXL3 template to avoid thought/channel marker leakage.
Architecture
| Property | Value |
|---|---|
| Architecture | Gemma4ForConditionalGeneration (model_type: gemma4) |
| Total Parameters | ~11.95B (dense) |
| Decoder Layers | 48 |
| Hidden Size | 3840 |
| Attention | 16 heads / 8 KV heads, hybrid sliding-window + global attention, p-RoPE |
| Vocabulary | 262,144 |
| Context Length | up to 131072 tokens (configured) |
Files
| File | Description |
|---|---|
model.safetensors |
EXL3 quantized weights (4bpw) |
config.json |
Model config |
processor_config.json |
Processor config |
tokenizer.json, tokenizer_config.json, chat_template.jinja, generation_config.json |
Standard |
Usage
import requests
url = "http://host:5000/v1/chat/completions"
headers = {"Authorization": "Bearer YOUR_API_KEY"}
payload = {
"model": "gemma-4-12B-it-abliterated-uncensored-exl3-4bpw",
"messages": [{"role": "user", "content": "Hello!"}],
"max_tokens": 512,
"enable_thinking": False,
}
response = requests.post(url, headers=headers, json=payload)
Notes
- License: Gemma (inherits the Gemma 4 license from the base model)
- Base Model: google/gemma-4-12B-it
- Modality: Text + Image + Audio (encoder-free / unified)
- Architecture: Gemma 4 12B Unified (dense, ~11.95B)
Disclaimer
Use is the responsibility of the user. Ensure your usage complies with applicable laws, platform rules, the Gemma 4 license terms, and your deployment requirements.