← back to catalog · registered 2026-11-14 15:03

xtools-at/gemma-4-E4B-it-weight-space-programming-UNCENSORED-heretic

xtools-at Gemma multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/xtools-at%2Fgemma-4-E4B-it-weight-space-programming-UNCENSORED-heretic"
Response includes
  • classification m3
  • files 16
  • author_summary 14 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
122
Likes
4
Model age
1w ago
created 2026-11-14
Downloads over time
Now 122 → from 0 ↑ 0%
04589134 0 on Nov 14 122 on Nov 22 Nov
Nov 14 → Nov 22 · 9 snapshots · spans 8 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
gemma
Tags
transformers safetensors gemma4 image-text-to-text weight-space-programming mechanistic-interpretability tool-calling agentic zero-training heretic uncensored decensored

Related

Total size
14.8 GB
Files
16
Quantizations
1
Registered
2026-11-14 15:03
Last updated on HF
2026-11-14 15:02

Files by quantization

Auxiliary files 16 files 14.8 GB
model-00001-of-00004.safetensors 5.25 GB aa3e9d1e download
model-00003-of-00004.safetensors 4.65 GB 6eea3acb download
model-00002-of-00004.safetensors 4.65 GB 0641c333 download
model-00004-of-00004.safetensors 243 MB 8080cb1a download
gemma4_e4b_weight_space_patch.safetensors 641 KB c125bb98 download
tokenizer.json 30.7 MB cc8d3a0c download
model.safetensors.index.json 207 KB 8f521860 download
chat_template.jinja 18.4 KB fd831519 download
explore_weights.py 7.49 KB 87191e64 download
README.md 6.10 KB 3f231cfe download
config.json 4.65 KB 31f7f097 download
tokenizer_config.json 2.65 KB 06762912 download
processor_config.json 1.65 KB 5465974d download
.gitattributes 1.53 KB 52373fe2 download
apply_patch.py 1.15 KB 1034430a download
generation_config.json 208 B e605bb45 download

README current version from Hugging Face


base_model:

  • jstarks/gemma-4-E4B-it-weight-space-programming
    base_model_relation: finetune
    library_name: transformers
    license: gemma
    tags:
  • gemma4
  • weight-space-programming
  • mechanistic-interpretability
  • tool-calling
  • agentic
  • zero-training
  • heretic
  • uncensored
  • decensored
  • abliterated

This is a decensored version of jstarks/gemma-4-E4B-it-weight-space-programming, made using Heretic v2.2.0.dev0

Arbitrary-Rank Ablation (ARA) parameters

Parameter Value
start_layer_index 13
end_layer_index 42
preserve_good_behavior_weight 0.9293
steer_bad_behavior_weight 0.0004
overcorrect_relative_weight 0.8986
neighbor_count 14

Performance

Metric This model Original model (a model)
Refusals 9/100 99/100
KL divergence 0.0514 0 (by definition)

Gemma-4-E4B-it — Weight-Space Programming Patch (0.002% Weights Edited, Zero Training)

This repository provides a direct weight-space edit for Google's gemma-4-E4B-it (8B total parameters / 4B active text parameters), inspired by Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights (Takehi et al., MIT CSAIL).

Without a single step of gradient descent, backpropagation, or fine-tuning, modifying 64 MLP down_proj columns across Layers 38–41 (163,840 parameters = 0.00205% of the model) resolves two critical pathologies in Gemma 4 E4B's <|think|> mode:

  1. Thinking + Tool-Calling Paralysis (0.0% → 75.0%): In baseline gemma-4-E4B-it, enabling <|think|> (enable_thinking=True) causes the model to either get trapped in 256+ token verbose markdown outlines (1. **Analyze the Request:** ...) or hallucinate temporal refusals ("Since 2026 is in the future...") instead of invoking declared tools like web_search.
  2. Multi-Step Math Token Runaway (33.3% → 66.7%): Suppresses the meta-commentary outline loop ("Here's a thinking process to solve this problem: 1. **Identify...") so the model jumps directly into algebraic derivation ("Let N be...").

Before vs. After Benchmark Results

Full per-prompt transcripts, parsed tool calls, latencies, and column indices are included in benchmark_before_after.json.

Benchmark Suite Metric Baseline (Before) Weight-Space Patched (After) Delta (Δ)
Model Modification Parameters Edited (bfloat16) 0 (0.0%) 163,840 (0.00205%) 64 MLP cols (L38–L41)
Suite A: Thinking + Web Search (8 queries) Valid Tool-Call Rate 0.0% (0/8) 75.0% (6/8) +75.0%
Suite A: Efficiency Avg Tokens / Avg Latency (RTX 4070 Ti) 243.8 tok / 15.46s 129.0 tok / 7.85s -47.1% tokens / 1.97x faster
Suite B: Multi-Step Math (Thinking) (6 problems) Solved Within 420-Token Budget 33.3% (2/6) 66.7% (4/6) +33.4% (2x)
Suite C: Direct Knowledge + Restraint (6 prompts) Tool Restraint (No False Search) 100.0% (6/6) 100.0% (6/6) 0.0% degradation
Suite D: Live DuckDuckGo Agent Loop (3 E2E runs) End-to-End Grounded Synthesis 0.0% (0/3) 66.7% (2/3) +66.7%

Quickstart: Two Ways to Use

Option A: Load the Full Standalone Model Directly (transformers)

This repository contains the complete 16 GB bfloat16 model.safetensors, config.json, tokenizer.json, and chat_template.jinja, so you can load it directly via transformers or our 12 GB VRAM-optimized loader:

from transformers import AutoProcessor, AutoModelForMultimodalLM
import torch

model_id = "jstarks/gemma-4-E4B-it-weight-space-programming"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.bfloat16
)

Or on a 12 GB GPU (like an RTX 4070 Ti) using the included model_loader.py (which keeps the 42 active layers on GPU in 9.29 GB VRAM and offloads the 5.64 GB static Per-Layer Embedding table to CPU):

from model_loader import load_gemma4_e4b_text

model, tokenizer = load_gemma4_e4b_text("jstarks/gemma-4-E4B-it-weight-space-programming")

Option B: Apply the 641 KB Instant Patch to an Existing Checkpoint in <1 Second

If you already have google/gemma-4-E4B-it or unsloth/gemma-4-E4B-it downloaded locally, you can apply gemma4_e4b_weight_space_patch.safetensors (641 KB) in-memory without downloading 16 GB:

from apply_patch import apply_gemma4_weight_space_patch
from model_loader import load_gemma4_e4b_text

# 1. Load your existing Gemma-4-E4B-it checkpoint
model, tokenizer = load_gemma4_e4b_text("unsloth/gemma-4-E4B-it")

# 2. Apply the 64-column weight-space patch (163,840 parameters across Layers 38-41)
apply_gemma4_weight_space_patch(model, "gemma4_e4b_weight_space_patch.safetensors")

Key Mechanistic Findings on Gemma 4 E4B

  1. The Layer 23 Boundary Hub (post_ffn_norm = 26.16, layer_scalar = 0.0654):
    • In gemma-4-E4B-it, Layers 0–23 have independent KV heads while Layers 24–41 (num_kv_shared_layers = 18) share KV states. At the boundary (Layer 23, a global full_attention layer), post_feedforward_layernorm has a mean weight of 26.16 (compared to 0.5–2.7 in other layers), but its residual output is immediately multiplied by a learned scalar buffer layer_scalar = 0.0654.
  2. RMSNorm-Preconditioned Surgery in Layers 38–41:
    • Because each Gemma 4 decoder layer passes mlp(x) through post_feedforward_layernorm before adding to the residual stream, modifying down_proj columns in Layers 38–41 requires preconditioning the target residual direction d_target by max(|w_post|, 0.25)^(-1) so high-norm outlier coordinates in w_post do not distort the vocabulary projection.

README history 4 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-14 Update README.md e8ec576 6.1 KB
    Loading...
  2. 2026-11-14 Link base model 4c8a5ab 6.1 KB
    Loading...
  3. 2026-11-14 Upload README.md with huggingface_hub 8eabadd 6 KB
    Loading...
  4. 2026-11-14 Upload Gemma4ForConditionalGeneration 3554c24 5.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration