license: apache-2.0
base_model: huihui-ai/Huihui-gemma-4-E2B-it-qat-q4_0-unquantized-abliterated-GGUF
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
- gguf
- gemma4
- abliterated
- uncensored
- imatrix
- llama.cpp
Gemma 4 E2B abliterated — Nocta GGUF (Q4_0 imatrix, PLE Q2_K, trimmed vocab)
Re-quantization of huihui-ai/Huihui-gemma-4-E2B-it-qat-q4_0-unquantized-abliterated
(BF16, QAT weights of google/gemma-4-E2B-it) made for the Nocta on-device companion app.
Sister repo: Vzdorrr/Nocta-Gemma-4-E4B-abliterated-GGUF.
| File | Size | What |
|---|---|---|
gemma-4-e2b-abliterated-imx-q40-ple2-v4.gguf |
1.88 GB | Q4_0 with importance matrix; per_layer_token_embd in Q2_K, token_embd in Q4_0; vocabulary trimmed to the app's scripts |
gemma-4-e2b-abliterated-mmproj-q5_0.gguf |
0.38 GB | vision projector, Q5_0 (from the BF16 projector) |
Recipe
llama-imatrix -m gemma-4-e2b-abliterated-bf16.gguf -f calib.txt -o g2imatrix.gguf -c 512
llama-quantize --imatrix g2imatrix.gguf \
--tensor-type per_layer_token_embd=q2_k --token-embedding-type q4_0 \
gemma-4-e2b-abliterated-bf16.gguf gemma-4-e2b-abliterated-imx-q40-ple2.gguf Q4_0
python3 vocab_trim.py gemma-4-e2b-abliterated-imx-q40-ple2.gguf gemma-4-e2b-abliterated-imx-q40-ple2-v4.gguf
Calibration text: app-style roleplay dialogues and prompts in the app's languages.
Q4_0 matches the QAT training of the weights and uses fast ARM kernels; the per-layer embedding tables are plain lookups, so Q2_K there costs almost nothing in chat quality.
Vocabulary trim. 262144 → 208339 tokens: tokens of scripts the app does not use (Devanagari, Arabic, Japanese kana, Thai and others) are removed.
Latin, Cyrillic, CJK ideographs, Korean, emoji and all special tokens are kept. On the app's texts the tokenization is identical to the untrimmed model.
Text in the removed scripts will tokenize poorly — use the untrimmed huihui model if you need those languages.
Measurements
| KL vs BF16 | Same top-1 | Bench (0–100), 3 seeds | |
|---|---|---|---|
| Q4_K_M, no imatrix | 0.059 | 88.4 % | 75 / 81 / 77 |
| this (KL before the vocab trim) | 0.070 | 87.3 % | 90 / 83 / 82 (avg 85) |
| for reference: our E4B | — | — | 81 / 95 / 80 |
The bench is a 30-turn Russian roleplay dialogue with memory, persona, robustness, vision, yes/no probes and format checks (llama.cpp, iOS simulator).
KL goes up a little, the dialogue bench goes up.
License
Apache 2.0, same as Gemma 4 and the abliterated base. Abliterated: refusals removed, so use responsibly.