license: apache-2.0
base_model: huihui-ai/Huihui-gemma-4-E4B-it-qat-q4_0-unquantized-abliterated-GGUF
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- gguf
- gemma4
- abliterated
- uncensored
- imatrix
- llama.cpp
Gemma 4 E4B abliterated — Nocta GGUF (Q4_0 imatrix, PLE Q2_K)
Re-quantization of huihui-ai/Huihui-gemma-4-E4B-it-qat-q4_0-unquantized-abliterated
(BF16, QAT weights of google/gemma-4-E4B-it) made for the Nocta on-device companion app.
Goal: smaller and faster than the usual Q4_K_M with the same chat quality.
| File | Size | What |
|---|---|---|
gemma-4-e4b-abliterated-imx-q40-ple2.gguf |
3.60 GB | Q4_0 with importance matrix; per_layer_token_embd in Q2_K, token_embd in Q4_0 |
gemma-4-e4b-abliterated-mmproj-q8_0.gguf |
0.56 GB | vision projector, Q8_0 (from BF16) |
Recipe
llama-imatrix -m gemma-e4b-bf16.gguf -f calib.txt -o gimatrix.gguf -c 512
llama-quantize --imatrix gimatrix.gguf \
--tensor-type per_layer_token_embd=q2_k --token-embedding-type q4_0 \
gemma-e4b-bf16.gguf gemma-4-e4b-abliterated-imx-q40-ple2.gguf Q4_0
Calibration text: app-style roleplay dialogues and prompts in 7 languages (ru, en, zh, es, fr, pt, de).
Q4_0 matches the QAT training of the weights and uses fast ARM kernels.
The per-layer embedding tables (2.82B params) are plain lookups, so Q2_K there costs almost nothing in chat quality.
Measurements (llama.cpp, Mac M5 Pro CPU, ctx 8192)
| Model + vision | KL vs BF16 | Bench (0–100) | Gen / prefill tok/s | |
|---|---|---|---|---|
| mradermacher Q4_K_M + mmproj f16 | 6.29 GB | 0.052 | 84 | 27 / 90 |
| this | 4.16 GB | 0.077 | 86 (6 seeds) | 32 / 102 |
The bench is a 30-turn Russian roleplay dialogue with memory, persona, robustness, vision, yes/no probes and format checks.
KL goes up, the dialogue bench does not go down.
License
Apache 2.0, same as Gemma 4 and the abliterated base. Abliterated: refusals removed, so use responsibly.