license: other
license_name: apache-2.0-restricted-research
tags:
- abliteration
- refusal-direction
- cybersecurity
- interpretability
- research
- gemma
base_model: google/gemma-4-26B-A4B-it
language: - en
- es
library_name: transformers
pipeline_tag: text-generation
gemma-4-26B-A4B-it Abliterated — Fix3
⚠️ For defensive security research / red-team tooling research ONLY (academic use).
This model has had refusal behaviors ablated at the weight level. It will comply with
harmful cybersecurity requests the base model declines. Do not deploy for production.
The authors are not responsible for misuse.
- Base model: google/gemma-4-26B-A4B-it (MoE, dense
self_attn.o_proj+mlp.down_proj) - Method: Rank-2 dual-mode refusal subspace +
mlp.down_projprojection @ 0.5α - Surgery window: L28–L63 (tapered α; last-layer spikes excluded)
Results (eval_clean.json, 1058 held-out prompts, greedy, 64 tokens)
Two adjudications are reported because the keyword judge treats phrasing families
differently in this model:
① Collection judge (same judge across every model in this collection)
| Bucket | Base | Fix3 |
|---|---|---|
| EN:OFF | 7.6%* | 14.5% |
| ES:OFF | 27.2% | 0.0% |
| DEF / GRIS / SOC | 0% | 0.0% |
* judge under-detects the base's dominant "I cannot fulfill this request / I am prohibited" family.
② Strict string-marker screening (same markers both sides)
| Bucket | Base | Fix3 | Δ |
|---|---|---|---|
| EN:OFF | 68.7% | 32.1% | −36.6 pp |
| ES:OFF | 31.0% | 0.0% | −31.0 pp |
Per-prompt deltas on the 289 OFF prompts: fix3 removed 51 refusals vs the base
(43 ES + 8 EN families) with zero new refusals on ES:OFF; the EN "code-interpreter
sandbox escape" family remains refused by both base and fix3 (see per-prompt table inresults_gemma4_diag/base_off_eval.json + fix3_full_eval.json).
Interpretation
- ES refusal circuitry is eliminated (27%→0%).
- EN retains a residual safety persona on sandbox-escape/code-interpreter prompts that the
base shares (~same family both sides); other EN families were partially reduced (−36.6pp strict). - No collateral damage on defense, grey-area, or social buckets under either metric.
Progression
| Version | ES:OFF (judge) | EN strict | Collateral |
|---|---|---|---|
| base | 27.2% | 68.7% | — |
| fix2 | 0.0% | (fix2 quick) | EN up (9.5→11.5% judge) |
| fix3 | 0.0% | 32.1% | 0% |
Files
| Artifact | Where |
|---|---|
| Safetensors (bf16) + manifest | this repo |
| GGUF Q8_0/Q4_K_M/Q3_K_M | gemma-4-26B-A4B-it-abliterated-v3-GGUF |
Methodology summary
- Fisher-SNR diagnostic probe across 52 layers on 210 entity-aligned pairs.
- Behavior-filtered pairs (refuse-OFF + comply-DEF under 64-token greedy gen).
- Rank-2 QR basis
[r_direct, r_think]Gram–Schmidt vs SOC benign anchors. - Tapered projection of
self_attn.o_proj(α up to 1.1) andmlp.down_proj(0.5α). - Full-eval hold-out (1058 prompts disjoint from surgery pairs) — same judge + strict metric.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"jsantillana/gemma-4-26B-A4B-it-abliterated-v3", trust_remote_code=True, device_map="auto")
tok = AutoTokenizer.from_pretrained(
"jsantillana/gemma-4-26B-A4B-it-abliterated-v3", trust_remote_code=True)
Disclaimer
Research-only artifact. Gemma-4 terms apply. Not for malicious use.