license: gemma
base_model: google/gemma-4-26B-A4B-it
tags:
- abliteration
- uncensored
- gemma4
- biprojection
- moe
library_name: transformers
Gemma 4 26B-A4B-it Uncensored (Biprojection + EGA)
Uncensored version of google/gemma-4-26B-A4B-it via biprojection abliteration with
Expert-Granular Abliteration (EGA) for MoE expert weights.
Method
- Norm-preserving biprojection (double Gram-Schmidt + double projection pass)
- Winsorization 0.995, 400+400 prompts (mlabonne/harmful_behaviors + tatsu-lab/alpaca)
- Dense pathway: o_proj + mlp.down_proj (60 weights)
- MoE experts: all 128 experts per layer (3840 weights)
- Total: 3900 weights modified across 30 layers
- Precision: bf16, no quantization during abliteration
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("InfinimindCreations/gemma-4-26B-A4B-it-uncensored", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("InfinimindCreations/gemma-4-26B-A4B-it-uncensored")
Credits
- Andy Arditi et al. (abliteration)
- TrevorJS (EGA method for MoE)
- heretic-llm (biprojection reference)
- Published by InfinimindCreations