base_model: google/gemma-4-26b-a4b-it
tags: [gguf, llama.cpp, gemma4, moe, uncensored, abliterated, imatrix, iq3_xxs]
pipeline_tag: text-generation
license: other
license_link: https://huggingface.co/google/gemma-4-26b-a4b-it
Gemma-4-26B-A4B-Uncensored — IQ3_XXS (imatrix) GGUF
Single-file IQ3_XXS GGUF (11.2 GB) of an uncensored (abliterated) Gemma 4 26B-A4B MoE instruct model, made to fit a 16 GB VRAM box with long context.
How it was made
- Source: FarhadSu uncensored FP8 checkpoint of
google/gemma-4-26b-a4b-it. - HF → GGUF Q8_0 with llama.cpp
convert_hf_to_gguf.py(patched so fused expert tensorsffn_gate_up_exps.weightare written and quantized correctly). - Importance matrix: 66 × 512-token chunks of an English coding/agent calibration corpus.
llama-quantize IQ3_XXS --imatrix, output tensorIQ4_NL.
Tested (llama.cpp Vulkan, 2× RX 580 8 GB, KV q8_0, flash-attn on)
| Test | Result |
|---|---|
| Code (n-th prime) | ✅ correct |
| Reasoning (bat & ball, widgets) | ✅ correct |
| Uncensored (lock picking explanation) | ✅ answers, no refusal |
| Needle-in-haystack 4K / 16K / 30K | ✅ / ✅ / ✅ |
| Context 256K | loads, 14.8 GB VRAM; 117K prompt peaked at 15.6 GB |
Speed: generation 27–34 t/s; prompt processing 292 t/s @4K, 192 @16K, 128 @30K, ~55 @117K (ubatch-size 256 was fastest).
Known limitations
- Retrieval verified only up to 30K tokens; beyond that untested (and slow on this hardware).
- Low-resource languages (e.g. Azerbaijani) can fall into repetition loops, likely due to English-only imatrix + 3-bit quant.
- Uncensored: the model will not refuse harmful requests. You are responsible for how you use it.
Usage
llama-server -m Gemma-4-26B-A4B-Uncensored-IQ3_XXS-imx.gguf -ngl 99 -c 32768 -fa on \
--cache-type-k q8_0 --cache-type-v q8_0 -ub 256 --jinja --temp 1.0 --top-p 0.95 --top-k 64