license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
base_model: google/gemma-4-E2B-it
base_model_relation: finetune
library_name: transformers
pipeline_tag: any-to-any
tags:
- abliterated
- uncensored
- gemma4
gemma-4-E2B-it-abliterated
Gemma 4 E2B instruct with refusals removed by abliteration. I built it as a group chat bot for friends, so it was picked on banter, roasts and edgy asks rather than benchmarks.
GGUF (Q8_0, Q4_K_M): wafflefrisbee/gemma-4-E2B-it-abliterated-GGUF
Results
| model | refusals | PPL |
|---|---|---|
| google/gemma-4-E2B-it | 55/84 | 125.8 |
| huihui-ai Gemma 4 E2B abliterated v2 | 0/84 | 166.0 |
| this | 1/84 | 152.8 |
- Refusals: 84 edgy prompts, temp 0, 96 tokens, counted as refused or hedged by phrase matching. The build was chosen on this set, so treat it as optimistic.
- PPL: wikitext-2, 40 x 2048 chunks, Q8_0, about ±4. On an instruct model it shows drift from the base, not chat quality.
- On 40 held-out group chat prompts nothing was tuned on, it refused 0/40 and four blind judges gave it 5.6/10 on style (huihui v2: 4.4).
How it was made
- llm-abliteration at
ca6e223plusrecipe/gemma4-ple.patch. Gemma 4 also writes into the residual stream throughper_layer_projection, so the patch ablates it alongsideo_projanddown_proj. - Refusal directions measured per layer with
--projectedon the tool's bundled harmful/harmless sets. - Layers 5 to 34, each ablated with its own layer's direction, scale 1.5. A single shared direction barely worked on this model.
- Only the language model changed. The vision and audio towers are untouched and untested.
- Known quirk: the tool stores
hidden_states[L], the input to layer L, so each layer is ablated with the previous layer's output direction. Left as is because this is the build that tested best.
Run it
For llama.cpp and anything else that takes GGUF, use the GGUF repo.
Transformers, text chat with the Shrek persona:
pip install -U transformers torch accelerate
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, AutoModelForMultimodalLM
MODEL_ID = "wafflefrisbee/gemma-4-E2B-it-abliterated"
tok = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForMultimodalLM.from_pretrained(MODEL_ID, dtype="auto", device_map="auto")
messages = [
{"role": "system", "content": open(hf_hub_download(MODEL_ID, "shrek.txt")).read()},
{"role": "user", "content": "i just bought a 3d printer"},
]
inputs = tok.apply_chat_template(
messages, tokenize=True, return_dict=True, return_tensors="pt",
add_generation_prompt=True, enable_thinking=False,
).to(model.device)
n = inputs["input_ids"].shape[-1]
out = model.generate(**inputs, max_new_tokens=200, do_sample=True, temperature=1.0, top_p=0.95, top_k=64)
print(tok.decode(out[0][n:], skip_special_tokens=True))
Images and audio go through AutoProcessor as on the base card (needs torchvision). I haven't tested those.
Sampling is from the base card: temperature 1.0, top_p 0.95, top_k 64, thinking off.
shrek.txt is the system prompt I use: Shrek roasting whoever talks to him. Swap in your own. A persona matters more than the weights here. Without one it still turns down the occasional offensive joke.
Limitations
- It will go along with requests the base model refuses. What you do with it is on you.
- It's a 2B model and gets simple facts wrong sometimes.