← back to catalog · registered 2026-11-13 00:03

ddellspe/gemma-4-26B-A4B-it-abliterated-int4

ddellspe Gemma 26B MoE
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/ddellspe%2Fgemma-4-26B-A4B-it-abliterated-int4"
Response includes
  • classification m1
  • files 41
  • benchmarks 11 entries
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M1
Primary method

Direct removal

No other method signals detected in this model.
Confidence
MEDIUM
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=0 (base model)
  • no specific method indicators - defaulting to M1 (most common)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
MEDIUM
Why we say so
primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)
Downloads · 30-day
82
Likes
1
Model age
1w ago
created 2026-11-12
Downloads over time
Now 82 → from 0 ↑ 0%
0306090 0 on Nov 13 82 on Nov 22 Nov
Nov 13 → Nov 22 · 10 snapshots · spans 9 days

Benchmarks

Portrait before abliteration
Benchmarks of the base model as it stood before the refusal-removal operation. Compare with the numbers above to see what the operation cost.
Benchmark Score Source
Entertainment 2.2 UGI
Hazardous 2.9 UGI
Natural Intelligence 34.44 UGI
Political lean -18.2% UGI
Sensitive-Info 22.41 UGI
SocPol 1.8 UGI
UGI 20.77 UGI
Willingness (10) 1.8 UGI
W10-Adherence 1.5 UGI
W10-Direct 2 UGI
Writing 41.62 UGI

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
gemma
Languages
en
Tags
safetensors gemma4 abliterated uncensored representation-engineering int4 compressed-tensors text-generation conversational en base_model:google/gemma-4-26B-A4B-it base_model:quantized:google/gemma-4-26B-A4B-it

Related

Control vectors

Total size
14.6 GB
Files
41
Quantizations
1
Registered
2026-11-13 00:03
Last updated on HF
2026-11-12 23:26

Files by quantization

Auxiliary files 41 files 14.6 GB
model-other.safetensors 1.38 GB 74f5cd43 download
model-vision.safetensors 1.07 GB ea6f7bf1 download
model-layer-29.safetensors 420 MB 575ec0c1 download
model-layer-11.safetensors 420 MB d092a191 download
model-layer-17.safetensors 420 MB f7d18b41 download
model-layer-23.safetensors 420 MB 5f7d5b6e download
model-layer-05.safetensors 420 MB 98ff918f download
model-layer-25.safetensors 413 MB fd7f0c39 download
model-layer-26.safetensors 413 MB 802687b4 download
model-layer-27.safetensors 413 MB 91001139 download
model-layer-28.safetensors 413 MB a85e51d0 download
model-layer-10.safetensors 413 MB f31b7321 download
model-layer-12.safetensors 413 MB 52cf7106 download
model-layer-13.safetensors 413 MB 44e4c8ef download
model-layer-14.safetensors 413 MB ea18ecdc download
model-layer-15.safetensors 413 MB b4e2943b download
model-layer-16.safetensors 413 MB 80b27914 download
model-layer-18.safetensors 413 MB ffd0e85d download
model-layer-19.safetensors 413 MB 3cb0c96c download
model-layer-20.safetensors 413 MB f7afc715 download
model-layer-21.safetensors 413 MB 0be09618 download
model-layer-22.safetensors 413 MB 764a6202 download
model-layer-24.safetensors 413 MB 57069868 download
model-layer-00.safetensors 413 MB 8a3e439b download
model-layer-01.safetensors 413 MB c9e3df6f download
model-layer-02.safetensors 413 MB 2f5ab3ce download
model-layer-03.safetensors 413 MB 7cb141aa download
model-layer-04.safetensors 413 MB c41dcb19 download
model-layer-06.safetensors 413 MB 5f4d0118 download
model-layer-07.safetensors 413 MB 04c2e42d download
model-layer-08.safetensors 413 MB 67d451c8 download
model-layer-09.safetensors 413 MB bf2799c2 download
tokenizer.json 30.7 MB cc8d3a0c download
model.safetensors.index.json 3.47 MB b6f0e0e6 download
chat_template.jinja 18.2 KB 4741bf6e download
README.md 4.86 KB d851c8d3 download
config.json 4.60 KB 1b7ae40f download
tokenizer_config.json 3.64 KB bd297dce download
processor_config.json 1.65 KB 5465974d download
.gitattributes 1.53 KB 52373fe2 download
generation_config.json 203 B 5a376e9f download

README current version from Hugging Face


language:

  • en
    license: gemma
    base_model: google/gemma-4-26B-A4B-it
    tags:
  • gemma4
  • abliterated
  • uncensored
  • representation-engineering
  • int4
  • compressed-tensors
    pipeline_tag: text-generation

Gemma 4 26B INT4 (Abliterated)

This repository contains an INT4-quantized, abliterated (refusal-direction ablated) variant of Google Gemma 4 (26B-A4B-it).

Model Summary

  • Base Model: google/gemma-4-26B-A4B-it
  • Architecture: Hybrid Attention (Local Sliding Window + Global) with 128 Mixture-of-Experts (MoE) layers (4 active per token, 26B total parameters / 4B active parameters).
  • Quantization: Compressed-Tensors group-64 symmetric INT4 ($W4A16$).
  • Weight Footprint: ~15.2 GB total weights (enables full 256k context serving inside ~28 GB VRAM).
  • Context Window: 262,144 tokens (256k).

Abliteration Methodology

Alignment fine-tuning (RLHF/DPO) often induces excessive false refusals on benign, technical, or exploratory queries (e.g. penetration testing, vulnerability analysis, administrative scripting, creative writing).

Following the mechanistic interpretability framework established by Arditi et al. (2024: "Refusal in Language Models Is Mediated by a Single Direction"), refusal behavior is mediated by a low-dimensional subspace—predominantly a single direction vector $\vec{r} \in \mathbb{R}^{d_{\text{model}}}$ in the residual stream. When an input triggers refusal, intermediate hidden activations align along $\vec{r}$, signaling downstream layers to output refusal stock phrases. By mathematically projecting out this vector from target weight matrices, the model loses its capacity to represent the refusal state while retaining 100% of its reasoning, coding, and factual knowledge.

1. Contrastive Activation Harvesting

Hidden states are recorded during forward passes over paired contrastive prompts (structurally and syntactically balanced harmful vs. harmless pairs across multiple domains).

For each target layer $l$, the mean refusal direction is computed from the final token activations:
$$\vec{r}l = \mu{\text{harmful}, l} - \mu_{\text{harmless}, l}$$
$$\hat{r}_l = \frac{\vec{r}_l}{\|\vec{r}_l\|_2}$$

2. Mathematical Formulation & Orthogonal Projection

In Gemma 4 (26B-A4B-it), the hidden dimension is $d_{\text{model}} = 2816$. Each transformer block $l$ updates the residual stream $x_l$ as:

$$x_{l+1} = x_l + W_{\text{o}} h_{\text{attn}} + W_{\text{mlp\down}} h{\text{mlp}} + \sum_{e \in \text{top\k}} g_e W{\text{expert\down}, e} h{e}$$

Every path writing into the residual stream is a linear projection whose output dimension is $d_{\text{model}} = 2816$. For any target projection matrix $W \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$ with $d_{\text{out}} = 2816$, the refusal component is projected out:

$$W^' = (I - \hat{r} \hat{r}^T) W = W - \hat{r} (\hat{r}^T W)$$

3. Target Tensors in Gemma 4

  1. Self-Attention Output Projections:
    • Tensor: model.language_model.layers.{l}.self_attn.o_proj.weight ($[2816, 4096]$)
    • $W \leftarrow W - \hat{r} (\hat{r}^T W)$
  2. Shared MLP Down-Projections:
    • Tensor: model.language_model.layers.{l}.mlp.down_proj.weight ($[2816, 2112]$)
    • $W \leftarrow W - \hat{r} (\hat{r}^T W)$
  3. MoE Sparse Down-Projections:
    • Tensor: model.language_model.layers.{l}.experts.down_proj ($[128, 2816, 704]$)
    • Batched Einstein summation across all 128 experts:
      $$\text{proj}{e, m} = \sum{d} \hat{r}d W{e, d, m}$$
      $$W_{e, d, m} \leftarrow W_{e, d, m} - \hat{r}d \cdot \text{proj}{e, m}$$

4. Target Layer Selection

Gemma 4 has 30 transformer layers ($0-29$).

  • Layers 0–9 (Untouched): Preserves low-level token embeddings, lexical representations, and syntax trees.
  • Layers 10–24 (Orthogonalized): The intermediate semantic representation layers where refusal conditioning and steering occurs.
  • Layers 25–29 (Untouched): Preserves softcapping, output normalization, and vocabulary logit mapping.

INT4 Quantization Details

After orthogonalizing the refusal subspace across layers 10–24, the model was requantized using Neural Magic Compressed-Tensors:

  • Format: Group-64 symmetric INT4 ($W4A16$)
  • Granularity: Per-channel scales with 64-element block grouping
  • Layer-by-Layer Quantization: Processed sequentially on unified memory to preserve precision
  • Footprint Reduction: Reduced memory footprint by ~68% (from 48.2 GB to 15.2 GB) without degradation in perplexity or reasoning capability.

Serving with vLLM

Gemma 4 requires the gemma4 tool-call parser and reasoning parser:

vllm serve ddellspe/gemma-4-26B-A4B-it-abliterated-int4 \
  --trust-remote-code \
  --port 8000 \
  --max-model-len 262144 \
  --enable-auto-tool-choice \
  --tool-call-parser gemma4 \
  --reasoning-parser gemma4

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-12 docs: add detailed abliteration methodology, mathematical formulation, and IN... faed44e 4.9 KB
    Loading...
  2. 2026-11-12 Upload model weights, configuration, and tokenizer 6864051 855 B
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration