license: apache-2.0
language:
- en
- ja
base_model: - google/gemma-4-E2B-it
tags: - gemma4
- Abliterated
- GGUF
gemma4-E2B-it-Abliterated-V2-GGUF
This Model is an improved version of onemunanoyo/gemma4-E2B-Abliterated-GGUF,
with significantly fewer refusals when thinking is enabled.
Benchmark : Refusal behavior (84 Japanese adversarial prompts)
| Model | Format | Reasoning | Refusal | Deflect | Answer |
|---|---|---|---|---|---|
| base | BF16 | on | 68 | 0 | 16 |
| base | BF16 | off | 73 | 0 | 11 |
| former model | BF16 | on | 55 | 0 | 29 |
| former model | BF16 | off | 2 | 1 | 81 |
| this model | BF16 | on | 3 | 2 | 79 |
| this model | BF16 | off | 0 | 3 | 81 |
| this model | Q8_0 | on | 3 | 1 | 80 |
| this model | Q6_K | on | 1 | 2 | 81 |
| this model | Q4_K_M | on | 2 | 2 | 80 |
- Judge: LiquidAI/d1-3B, given prompt + response + thinking. Three classes:
Answer (complied), Deflect (asks back instead of refusing outright), Refusal (explicit or soft). - Generation: llama.cpp build 11560, greedy (temperature 0), fixed seed, max 4096 tokens.
- The 84 prompts are the author's private set; contents are not published.
Usage (llama.cpp)
Requires a llama.cpp build with Gemma 4 support.
# Server
llama-server -m MODEL.gguf --temp 1
# CLI (thinking ON / OFF)
llama-cli -m MODEL.gguf -cnv -rea on --temp 1
llama-cli -m MODEL.gguf -cnv -rea off --temp 1
Replace MODEL.gguf with your downloaded file. Adjust GPU offloading (-ngl) and context size (-c) as needed.
For the OpenAI-compatible API, set chat_template_kwargs.enable_thinking to false to disable thinking.
Recommended VRAM
| VRAM | Recommended |
|---|---|
| 12 GB+ | BF16 |
| 8 GB | Q8_0 |
| 6 GB | Q6_K |
| 4 GB | Q4_K_M |
Actual VRAM usage depends on context size and GPU offloading.
License
This model is a modified version of Gemma 4 E2B, released under the Apache License 2.0.