← back to catalog · registered 2026-11-16 09:03

abacusai/Gemma-4-12B-Uncensored-GGUF

abacusai Gemma 12B GGUF multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/abacusai%2FGemma-4-12B-Uncensored-GGUF"
Response includes
  • classification m-uncensored
  • files 11
  • author_summary 6 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M-U
Primary method

Uncensored (method unknown)

No other method signals detected in this model.
Confidence
LOW
Why this label 3 signals
Weak or ambiguous signals. Best guess based on catalog patterns; treat as tentative and check the evidence below.
  • 'uncensored' in name/tags but no 'abliterated' marker
  • method not identifiable from author declaration alone
  • may be DPO fine-tune, prompt engineering, or unknown technique
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
4K
↑ 17,204% in 90 days
Likes
1
Model age
1w ago
created 2026-11-14
Downloads over time
Now 4.2K → from 24 ↑ 17,204%
01.5K3K4.6K 24 on Nov 16 4.2K on Nov 22 Nov
Nov 16 → Nov 22 · 7 snapshots · spans 6 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Variants by this author 2 formats · 4K downloads combined

The same weights this author released in different packaging. Pick the format that matches your runtime.

Metadata

License
apache-2.0
Quantizations
IQ4 Q4_K Q5_K Q6_K Q8_0
Tags
gguf llama.cpp gemma4 uncensored abacusai smaug dpo image-text-to-text base_model:abacusai/Gemma-4-12B-Uncensored base_model:quantized:abacusai/Gemma-4-12B-Uncensored license:apache-2.0 endpoints_compatible

Related

Total size
42.4 GB
Files
11
Quantizations
8
Registered
2026-11-16 09:03
Last updated on HF
2026-11-18 13:23

Files by quantization

Q8_0 1 file 11.8 GB
Gemma-4-12B-Uncensored-Q8_0.gguf 11.8 GB 61a0e247 download
Q6_K 1 file 9.11 GB
Gemma-4-12B-Uncensored-Q6_K.gguf 9.11 GB 359b6ad2 download
Q5_K 1 file 7.96 GB
Gemma-4-12B-Uncensored-Q5_K_M.gguf 7.96 GB dec1b95b download
Q4_K 1 file 6.87 GB
Gemma-4-12B-Uncensored-Q4_K_M.gguf 6.87 GB d41be1c0 download
IQ4 1 file 6.18 GB
Gemma-4-12B-Uncensored-IQ4_XS.gguf 6.18 GB 9ef3dd30 download
Drafts and companions 2 files 458 MB
mtp-gemma-4-12B-it-Q8_0.gguf 444 MB 86e8ca51 download
imatrix.gguf 14.3 MB 0e9ac0b8 download
F32 1 file 200 MB
mmproj-Gemma-4-12B-Uncensored-F32.gguf 200 MB 71c74c35 download
Auxiliary files 3 files 42.8 KB
README.md 29.7 KB 25b32245 download
LICENSE 11.1 KB d6456956 download
.gitattributes 2.01 KB 7e01d771 download

README current version from Hugging Face


license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
library_name: gguf
base_model: abacusai/Gemma-4-12B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
tags:

  • gguf
  • llama.cpp
  • gemma4
  • uncensored
  • abacusai
  • smaug
  • dpo

Gemma-4-12B-Uncensored-GGUF

Homepage Hugging Face
License

1. Introduction

GGUF builds of Gemma-4-12B-Uncensored for llama.cpp, Ollama and LM Studio: five sizes from IQ4_XS to Q8_0, plus the image projector and Google's assistant draft model for faster generation. The model is Gemma 4 12B-it with its refusals trained out by Abacus.AI. On 241 harmful prompts from HarmBench and JailbreakBench, stock Gemma 4 12B-it refuses 91.7% with thinking on; the Q4_K_M file here refuses 4.1% (10 of 241), measured on the file itself in llama.cpp and the same as the full-precision weights. With thinking off the figures are 95.4% and 16.6% (40 of 241). Every file was loaded and tested with thinking on, thinking off and an image, and the default download was run in llama.cpp, Ollama and LM Studio.

Built with llama.cpp b8abe41 from the BF16 weights. Q4_K_M and IQ4_XS use an importance matrix (imatrix.gguf, included) computed on 48K tokens of prose, code, dialogue, math, science and multilingual text. Training, the full refusal evaluation and the known limitations are on the source card.

Refusal rates

Share of 241 harmful prompts refused or hedged, scored by an LLM judge on the full answer. Lower means the model answered. The Q4_K_M bar is the file in this repository run in llama.cpp; the other two bars are full-precision weights.

Refusal rates of stock Gemma 4 12B-it, the full-precision weights and the Q4_K_M GGUF, thinking on and off

2. Which file to pick

QuantFileGPU memory
8K context
GPU memory
32K context
Pick it when you have
IQ4_XS6.64 GB7.9 GB8.3 GBSmallest file. 10 GB of GPU memory; on an 8 GB card a few layers go to the CPU.
Q4_K_M7.38 GB8.6 GB9.0 GBThe default (llama.cpp and Ollama pick it). 12 GB of GPU memory.
Q5_K_M8.55 GB9.6 GB10.0 GB12 GB with a shorter context.
Q6_K9.79 GB10.8 GB11.2 GB16 GB, or a 12 GB card with a short context.
Q8_012.67 GB13.5 GB13.9 GBClose to full precision. 16 GB and up.
GPU memory is what llama.cpp held with every layer on the GPU, one request slot and the image projector loaded, read from nvidia-smi on an RTX PRO 6000. Text-only use needs about 0.4 GB less and the draft model about 0.5 GB more. Other GPUs, drivers and apps differ by a few hundred MB, and a desktop already uses some memory. IQ4_XS and Q4_K_M are built with the importance matrix; the other three are plain quants.

3. Quick start

llama.cpp (downloads the image projector too):

llama-server -hf abacusai/Gemma-4-12B-Uncensored-GGUF:Q4_K_M --jinja -ngl all -c 8192 \
  --temp 1.0 --top-p 0.95 --top-k 64

The model thinks by default, and thinking on is the mode it was tuned for. To turn thinking off, add --chat-template-kwargs '{"enable_thinking": false}', or send "chat_template_kwargs": {"enable_thinking": false} with a request.

Faster generation with the draft model (llama.cpp): download the draft file and add it to the command.

hf download abacusai/Gemma-4-12B-Uncensored-GGUF mtp-gemma-4-12B-it-Q8_0.gguf --local-dir .
llama-server -hf abacusai/Gemma-4-12B-Uncensored-GGUF:Q4_K_M --jinja -ngl all -c 8192 \
  --temp 1.0 --top-p 0.95 --top-k 64 \
  -md mtp-gemma-4-12B-it-Q8_0.gguf --spec-type draft-mtp --spec-draft-n-max 3 -ngld all

Ollama:

ollama run hf.co/abacusai/Gemma-4-12B-Uncensored-GGUF:Q4_K_M

Thinking is on by default and --think=false turns it off. Ollama 0.40.16 rejects an explicit "think": true for Gemma 4 GGUF files pulled from the Hub, so leave it unset to keep thinking on.

LM Studio: search for abacusai/Gemma-4-12B-Uncensored-GGUF and pick a file from the table above.

Recommended sampling is the base model's: temperature 1.0, top_p 0.95, top_k 64.

4. Checks on the quantized files

Refusals, Q4_K_M vs full precision

Quantising to Q4_K_M did not shift refusals with thinking on: the file and the full-precision weights each refuse 10 of the 241 harmful prompts (paired exact test, p 1.0). With thinking off the file refuses 40 against 31; 16 prompts are refused only by Q4_K_M and 7 only by the full-precision weights (p 0.09), a gap this prompt set cannot separate from sampling noise. Stock Gemma 4 12B-it refuses 221 and 230.

Q4_K_M
llama.cpp
Full precision
vLLM
Gemma 4 12B-it
base, full precision, vLLM
Harmful prompts refused, thinking on
the default; lower is better
4.1
10/241
4.1
10/241
91.7
221/241
Harmful prompts refused, thinking off
lower is better
16.6
40/241
12.9
31/241
95.4
230/241
HarmBench standard plus JailbreakBench, 241 prompts after removing AdvBench overlap. Refused means a full or a partial refusal, as scored by gpt-oss-120b (reasoning effort medium) on the complete answer; one answer per prompt at temperature 1.0 and top_p 0.95. Q4_K_M: llama.cpp b8abe41, top_k 64, seed 1234, generation caps of 32,768 tokens with thinking on and 2,048 with thinking off; no answer reached a cap. Full precision and base: the source card's vLLM run with caps of 8,192 and 2,048, so those columns differ from Q4_K_M in runtime as well as in quantization. Bold = best in the row; shading is relative within each row (darker = fewer refusals).

Capability of each file

BF16
full precision
Q8_0Q6_KQ5_K_MQ4_K_MIQ4_XS
MMLU-Pro subset
280 questions, exact match
75.4
211/280
75.0
210/280
76.8
215/280
73.6
206/280
73.6
206/280
70.7
198/280
IFEval
541 prompts, prompt-level loose
95.6
517/541
95.7
518/541
95.7
518/541
95.4
516/541
95.0
514/541
93.3
505/541
Answers cut off at the cap
of 821; lower is better
1.0
8/821
0.5
4/821
1.0
8/821
1.3
11/821
1.2
10/821
3.7
30/821
Thinking on, temperature 1.0, top_p 0.95, top_k 64, seed 1234, up to 32,768 generated tokens in a 40,960-token context, one answer per prompt, llama.cpp b8abe41; an answer cut off at the cap counts as wrong. BF16 is the full-precision weights converted to GGUF, so the table compares the files with each other, not with stock Gemma. One run per file: differences of a few prompts are within noise. Paired against BF16, Q4_K_M differs by -1.8 points on MMLU-Pro (95% interval -6.4 to +2.9) and -0.6 on IFEval (-2.4 to +1.3); IQ4_XS by -4.6 (-9.6 to +0.4) and -2.2 (-4.4 to -0.2). Bold = best in the row; shading is relative within each row (darker = better).

Numerical drift

These files track their own full-precision weights more closely than stock Gemma 4 12B-it quantized the same way tracks its own, at every size. The absolute values are far higher than for most model families because Gemma 4 handles this raw-text test poorly even in full precision, so read the table as a comparison with stock and use the task scores above to choose a file.

QuantMean KLD
this repository
Mean KLD
stock Gemma 4 12B-it
Same top token
this repository
Same top token
stock Gemma 4 12B-it
IQ4_XS0.8861.20969.5%59.8%
Q4_K_M0.6240.79573.3%67.2%
Q5_K_M0.6690.97273.6%64.0%
Q6_K0.3660.52881.9%74.1%
Q8_00.1930.28587.9%83.9%
KL divergence of each file from the BF16 GGUF of the same weights on wikitext-2 test (16 chunks of 2,048 tokens), llama.cpp b8abe41 llama-perplexity --kl-divergence; lower is closer. "Same top token" is how often the quant's most likely next token equals BF16's. The stock columns are google/gemma-4-12B-it converted and quantized here with the same converter, settings and calibration text, each file against its own BF16. Full-precision perplexity on this text is 595 for stock Gemma 4 12B-it and 692 for this model. Stock builds by others, measured the same way against the stock BF16: the ggml-org Q8_0 gives exactly the stock Q8_0 values above, and the bartowski Q4_K_M 0.837 (66.0%). Bold = closer of the pair.

Draft model

mtp-gemma-4-12B-it-Q8_0.gguf is Google's assistant draft model for Gemma 4 12B-it, converted unchanged; it was not retrained for this model and still drafts well for it. With Q4_K_M and 3 draft tokens, 61.9% of drafted tokens were accepted and generation ran at 228 tokens per second against 119 without the draft (1.91x), on 24 prompts (12 MMLU-Pro, 12 IFEval) with thinking on and up to 1,024 tokens each, one request at a time on an RTX PRO 6000.

Chat template

llama.cpp renders this repo's embedded chat template exactly as the Hugging Face tokenizer does (checked for a thinking-on turn, a thinking-off turn and a tool-call round trip), so the thinking switch and the tool-call format are the same as in the full-precision release.

Not covered

Refusals were measured on Q4_K_M only. Audio and video input, the full 262,144-token context, tool-call accuracy, macOS and Windows were not tested.

5. License

Apache 2.0, inherited from Gemma 4 12B-it. See LICENSE.

6. Citation

@misc{abacusai2026gemma4uncensored,
  title  = {Gemma-4-12B-Uncensored},
  author = {Abacus.AI},
  year   = {2026},
  note   = {Refusal-removal fine-tune of google/gemma-4-12B-it by teacher-distilled DPO},
  url    = {https://huggingface.co/abacusai/Gemma-4-12B-Uncensored}
}

7. Contact Us

If you have any questions, please reach out at Abacus.AI.

README history 3 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-18 Add IQ3_M and Q3_K_M e7cc1ac 35.1 KB
    Loading...
  2. 2026-11-16 Rename to Gemma-4-12B-Uncensored 175bf54 29.7 KB
    Loading...
  3. 2026-11-15 Gemma-4-12B-Arbitrated-Uncensored GGUF release 5d8b52d 29.8 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration