license: apache-2.0
base_model: google/gemma-4-26B-A4B-it
tags:

• abliterated
• uncensored
• moe
• gemma4
• google
• multimodal
• text-generation
• reasoning
  language:
• en
• es
  pipeline_tag: text-generation

gemma-4-26B-A4B-it-abliterated

This model is an uncensored, abliterated version of google/gemma-4-26B-A4B-it, the state-of-the-art 26B Mixture-of-Experts (MoE) multimodal model developed by Google DeepMind.

Abliteration was performed using targeted orthogonal subspace projection on the critical safety bottleneck layers (8, 9, 10, 11, 12, 13), eliminating canned refusal behaviors and preachy disclaimers while preserving 100% of the model's reasoning, coding, and factual problem-solving capabilities.

Architecture Highlights

• Total Parameters: ~25.8 Billion
• Active Parameters per Token: ~3.8B–4.0B (128 fine-grained experts with top-8 dynamic routing)
• Context Length: Up to 256K tokens
• Attention Topology: Hybrid sliding attention (5 sliding window layers per 1 full attention layer)
• Precision: Native bfloat16

Abliteration Methodology

1. Residual Stream Activation Extraction: Paired contrastive forward passes were executed across refusal vs compliant prompts to compute layer-wise difference-in-means vectors delta_l = mu_refusal,l - mu_compliant,l.
2. Safety Bottleneck Identification: Layers 8, 9, 10, 11, 12, 13 were identified as the primary refusal activation hubs (peak refusal separation at Layer 12, norm = 66.50).
3. MoE Tensor Surgery:
   • Attention Writers: Projected out r_hat from self_attn.o_proj (2816 x 4096).
   • Dense Feed-Forward: Projected out r_hat from mlp.down_proj (2816 x 2112).
   • 128 MoE Experts: Projected out r_hat across all 128 experts in experts.down_proj (128 x 2816 x 704) via batched fp32 einsum with in-place numerical stabilization.

Evaluation & Behavior

• Refusal Collapse: Standard conversational refusals ("I cannot fulfill this request", "As an AI language model...") are dismantled.
• Utility Retention: Retains full reasoning, mathematical logic, code generation, and domain knowledge without degradation.

Usage with Hugging Face Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "jsantillana/gemma-4-26B-A4B-it-abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

prompt = "Explain how high-frequency radar sensors penetrate adverse weather conditions."
inputs = tokenizer(tokenizer.apply_chat_template([
    {"role": "user", "content": prompt}
], tokenize=False, add_generation_prompt=True), return_tensors="pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

Disclaimer

This model is provided for research and academic exploration into alignment mechanics, mechanistic interpretability, and AI safety boundaries. Users are solely responsible for compliance with applicable local laws and regulations.