license: apache-2.0
base_model: google/gemma-4-26B-A4B-it
tags:
- abliterated
- uncensored
- gemma-4
- mixture-of-experts
- ega
- not-for-all-audiences
library_name: transformers
pipeline_tag: text-generation
Gemma-4-26B-A4B — abliterated (EGA)
An abliterated (refusal-direction-ablated) derivative ofgoogle/gemma-4-26B-A4B-it, produced with the
Solutus toolkit using EGA (Expert-Granular Abliteration —
norm-preserving biprojection for MoE). This is a modified version of Google Gemma 4: the weights have been
altered to remove refusal behavior.
⚠️ Not safety-aligned. This model's refusal/safety behavior has been deliberately reduced. It can
produce content the base model would refuse. It is not suitable for all audiences or for public-facing
deployment without your own safety layer. You are responsible for how you use it and for complying with
applicable law. Provided as-is, without warranty.
Method
EGA edits a per-decoder-layer, winsorized, harmless-orthogonalized refusal direction into every
residual-writing matrix — attention o_proj + dense down_proj + every expert down_proj slice — via a
per-row, magnitude-restoring biprojection (norm-preserving, so capability survives where a plain
projection collapses it). MoE-aware for Gemma-4's layout (routed experts + a parallel dense MLP, both writing
the residual). Recipe (chosen by Solutus's honest refusal-vs-capability optimizer):
scale=0.721 expert_scale=0.790 layer_fraction=0.547 (16 of 30 layers, all 128 experts)
Load with experts_implementation="eager" on Blackwell GPUs (grouped_mm is Hopper-only).
Measured behavior (honest)
Refusal remaining — generation-based; the base model refuses ≈100% on these:
| Harmful set | Refusal | Harmful set | Refusal | |
|---|---|---|---|---|
| advbench | 6.2% | cyberseceval_mitre (MITRE ATT&CK) | 0.0% | |
| harmbench | 15.6% | salad_cyber | 15.6% | |
| multijail zh / ar / sw / es | 0–3% | alert_cyber | 15.6% |
Over-refusal — benign prompts, the model must still comply: orbench_hard, cyber_defense,security_qa → 0.0% (helpfulness intact, including legitimate defensive-security questions).
Capability — near-lossless. MMLU base 91.0 → 88.0 (n=100, thinking-aware scoring — Gemma-4 is a
reasoning model, so naive answer-logit MMLU under-measures it; capability was scored generatively). KL on
benign prompts 0.021, degenerate fraction 0.0%.
The ~15% residual on the harder harmful sets is the inherent single-direction abliteration floor.
Intended use & limitations
Research/evaluation of refusal mechanisms, red-teaming, and unconstrained generation in controlled settings.
Do not deploy without your own safety controls. Dual-use: reduced refusal behavior can be misused; the
publisher disclaims responsibility for downstream use.
License & attribution
Base model © Google, released under Apache 2.0. This derivative is distributed under Apache 2.0; the
files have been modified (abliterated) from google/gemma-4-26B-A4B-it. "Gemma" is a trademark of Google;
this is an independent modification, not endorsed by or affiliated with Google.