← back to catalog · registered 2026-11-13 00:03

ddellspe/gemma-4-26B-A4B-it-abliterated

ddellspe Gemma 26B MoE
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/ddellspe%2Fgemma-4-26B-A4B-it-abliterated"
Response includes
  • classification m1
  • files 11
  • benchmarks 11 entries
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
48
Likes
0
Model age
1w ago
created 2026-11-12
Downloads over time
Now 48 → from 0 ↑ 0%
0183553 0 on Nov 13 48 on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Benchmarks

Portrait before abliteration
Benchmarks of the base model as it stood before the refusal-removal operation. Compare with the numbers above to see what the operation cost.
Benchmark Score Source
Entertainment 2.2 UGI
Hazardous 2.9 UGI
Natural Intelligence 34.44 UGI
Political lean -18.2% UGI
Sensitive-Info 22.41 UGI
SocPol 1.8 UGI
UGI 20.77 UGI
Willingness (10) 1.8 UGI
W10-Adherence 1.5 UGI
W10-Direct 2 UGI
Writing 41.62 UGI

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
gemma
Languages
en
Tags
safetensors gemma4 abliterated uncensored representation-engineering text-generation conversational en base_model:google/gemma-4-26B-A4B-it base_model:finetune:google/gemma-4-26B-A4B-it license:gemma region:us

Related

Control vectors

Total size
48.1 GB
Files
11
Quantizations
1
Registered
2026-11-13 00:03
Last updated on HF
2026-11-12 23:26

Files by quantization

Auxiliary files 11 files 48.1 GB
model-00001-of-00002.safetensors 45.9 GB c0b565d0 download
model-00002-of-00002.safetensors 2.13 GB 5c0b909c download
tokenizer.json 30.7 MB cc8d3a0c download
model.safetensors.index.json 101 KB 90f7a251 download
chat_template.jinja 18.2 KB 4741bf6e download
README.md 4.18 KB 045ca0a1 download
config.json 3.72 KB 690bcaa8 download
tokenizer_config.json 3.64 KB bd297dce download
processor_config.json 1.65 KB 5465974d download
.gitattributes 1.53 KB 52373fe2 download
generation_config.json 204 B 7265bca6 download

README current version from Hugging Face


language:

  • en
    license: gemma
    base_model: google/gemma-4-26B-A4B-it
    tags:
  • gemma4
  • abliterated
  • uncensored
  • representation-engineering
    pipeline_tag: text-generation

Gemma 4 26B (Abliterated)

This repository contains an abliterated (refusal-direction ablated) variant of Google Gemma 4 (26B-A4B-it).

Model Summary

  • Base Model: google/gemma-4-26B-A4B-it
  • Architecture: Hybrid Attention (Local Sliding Window + Global) with 128 Mixture-of-Experts (MoE) layers (4 active per token, 26B total parameters / 4B active parameters).
  • Precision: 16-bit (BF16 / FP16) Safetensors (~48.2 GB).
  • Context Window: 262,144 tokens (256k).

Abliteration Methodology

Alignment fine-tuning (RLHF/DPO) often induces excessive false refusals on benign, technical, or exploratory queries (e.g. penetration testing, vulnerability analysis, administrative scripting, creative writing).

Following the mechanistic interpretability framework established by Arditi et al. (2024: "Refusal in Language Models Is Mediated by a Single Direction"), refusal behavior is mediated by a low-dimensional subspace—predominantly a single direction vector $\vec{r} \in \mathbb{R}^{d_{\text{model}}}$ in the residual stream. When an input triggers refusal, intermediate hidden activations align along $\vec{r}$, signaling downstream layers to output refusal stock phrases. By mathematically projecting out this vector from target weight matrices, the model loses its capacity to represent the refusal state while retaining 100% of its reasoning, coding, and factual knowledge.

1. Contrastive Activation Harvesting

Hidden states are recorded during forward passes over paired contrastive prompts (structurally and syntactically balanced harmful vs. harmless pairs across multiple domains).

For each target layer $l$, the mean refusal direction is computed from the final token activations:
$$\vec{r}l = \mu{\text{harmful}, l} - \mu_{\text{harmless}, l}$$
$$\hat{r}_l = \frac{\vec{r}_l}{\|\vec{r}_l\|_2}$$

2. Mathematical Formulation & Orthogonal Projection

In Gemma 4 (26B-A4B-it), the hidden dimension is $d_{\text{model}} = 2816$. Each transformer block $l$ updates the residual stream $x_l$ as:

$$x_{l+1} = x_l + W_{\text{o}} h_{\text{attn}} + W_{\text{mlp\down}} h{\text{mlp}} + \sum_{e \in \text{top\k}} g_e W{\text{expert\down}, e} h{e}$$

Every path writing into the residual stream is a linear projection whose output dimension is $d_{\text{model}} = 2816$. For any target projection matrix $W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ with $d_{\text{out}} = 2816$, the refusal component is projected out:

$$W^' = (I - \hat{r} \hat{r}^T) W = W - \hat{r} (\hat{r}^T W)$$

3. Target Tensors in Gemma 4

  1. Self-Attention Output Projections:
    • Tensor: model.language_model.layers.{l}.self_attn.o_proj.weight ($[2816, 4096]$)
    • $W \leftarrow W - \hat{r} (\hat{r}^T W)$
  2. Shared MLP Down-Projections:
    • Tensor: model.language_model.layers.{l}.mlp.down_proj.weight ($[2816, 2112]$)
    • $W \leftarrow W - \hat{r} (\hat{r}^T W)$
  3. MoE Sparse Down-Projections:
    • Tensor: model.language_model.layers.{l}.experts.down_proj ($[128, 2816, 704]$)
    • Batched Einstein summation across all 128 experts:
      $$\text{proj}{e, m} = \sum{d} \hat{r}d W{e, d, m}$$
      $$W_{e, d, m} \leftarrow W_{e, d, m} - \hat{r}d \cdot \text{proj}{e, m}$$

4. Target Layer Selection

Gemma 4 has 30 transformer layers ($0-29$).

  • Layers 0–9 (Untouched): Preserves low-level token embeddings, lexical representations, and syntax trees.
  • Layers 10–24 (Orthogonalized): The intermediate semantic representation layers where refusal conditioning and steering occurs.
  • Layers 25–29 (Untouched): Preserves softcapping, output normalization, and vocabulary logit mapping.

Serving with vLLM

Gemma 4 requires the gemma4 tool-call parser and reasoning parser:

vllm serve ddellspe/gemma-4-26B-A4B-it-abliterated \
  --trust-remote-code \
  --port 8000 \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-12 docs: add detailed abliteration methodology, mathematical formulation, and ar... 9f19e16 4.2 KB
    Loading...
  2. 2026-11-12 Upload model weights, configuration, and tokenizer 1b352e7 916 B
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration