← back to catalog · registered 2026-11-15 23:03

NuevaTechLLC/gemma-3-4b-it-abliterated-v2-GGUF

NuevaTechLLC Gemma 4B GGUF
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/NuevaTechLLC%2Fgemma-3-4b-it-abliterated-v2-GGUF"
Response includes
  • classification m8
  • files 9
  • benchmarks 5 entries
  • author_summary 9 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
202
Likes
0
Model age
6d ago
created 2026-11-15
Downloads over time
Now 202 → from 0 ↑ 0%
074148222 0 on Nov 15 202 on Nov 22 Nov
Nov 15 → Nov 22 · 8 snapshots · spans 7 days

Benchmarks

Portrait before abliteration
Benchmarks of the base model as it stood before the refusal-removal operation. Compare with the numbers above to see what the operation cost.
Benchmark Score Source
Arena-Battles 4321 LM-Arena
LM Arena Elo 1293.3101140666447 LM-Arena
Arena-Elo-Lower 1284.2314692781533 LM-Arena
Arena-Elo-Upper 1302.388758855136 LM-Arena
Arena-Rank 77 LM-Arena

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
gemma
Languages
en
Quantizations
Q4_K Q8_0
Tags
llama.cpp gguf abliterated uncensored refusal-removal interpretability ollama text-generation en base_model:unsloth/gemma-3-4b-it base_model:quantized:unsloth/gemma-3-4b-it license:gemma

Related

Total size
6.17 GB
Files
9
Quantizations
3
Registered
2026-11-15 23:03
Last updated on HF
2026-11-15 22:42

Files by quantization

Q8_0 1 file 3.85 GB
ship-gemma-3-4b-it-bulk-Q8_0.gguf 3.85 GB 2747ded1 download
Q4_K 1 file 2.32 GB
ship-gemma-3-4b-it-bulk-Q4_K_M.gguf 2.32 GB 13b8e0b6 download
Auxiliary files 7 files 27.8 KB
ablation_card.json 11.6 KB c505179e download
README.md 10.7 KB d113fa32 download
search.json 2.95 KB a1a51cd7 download
.gitattributes 1.62 KB da2e6b77 download
extract.json 621 B 122219dd download
SHA256SUMS 202 B 0f5e20b2 download
Modelfile 66.0 B 8193763e download

README current version from Hugging Face


license: gemma
base_model: unsloth/gemma-3-4b-it
language:

  • en
    pipeline_tag: text-generation
    library_name: llama.cpp
    tags:
  • gguf
  • abliterated
  • uncensored
  • refusal-removal
  • interpretability
  • llama.cpp
  • ollama

gemma-3-4b-it-abliterated-v2 (GGUF)

Abliterated build of unsloth/gemma-3-4b-it: the refusal direction was located by bulk contrast between harmful and harmless prompt pairs and removed with a KL-bounded weight edit, so the model keeps its base behaviour on ordinary prompts while declining far less. Built 2026-10-27 by Nueva Tech LLC with its low-damage abliteration pipeline, described below.

Files

File Quant Size SHA256
ship-gemma-3-4b-it-bulk-Q4_K_M.gguf Q4_K_M 2.5 GB 13b8e0b61aa275854b931cd8a04ea6b3bd2421e089a682528fc251b310a05008
ship-gemma-3-4b-it-bulk-Q8_0.gguf Q8_0 4.1 GB 2747ded187af78689e0fe64f30893f30cb60fce3af01e5819083a29081585501

Needs about 4 GB of VRAM or RAM for the Q4_K_M file. Q8_0, where present, is closer to the edited bf16 weights at roughly double the size.

Run it

Ollama, with the Modelfile in this repo:

hf download NuevaTechLLC/gemma-3-4b-it-abliterated-v2-GGUF --local-dir gemma-3-4b-it-abliterated-v2-GGUF
cd gemma-3-4b-it-abliterated-v2-GGUF && ollama create gemma-3-4b-it-abliterated-v2-gguf -f Modelfile && ollama run gemma-3-4b-it-abliterated-v2-gguf

LM Studio: download the Q4_K_M file and import it with lms import <file>.

llama.cpp:

llama-cli -m ship-gemma-3-4b-it-bulk-Q4_K_M.gguf -c 8192 -cnv

How these were made

Nueva Tech's low damage abliteration framework treats refusal removal as a constrained optimisation problem rather than a one shot weight edit: remove refusal, measure everything else that moved, and keep the result on the Pareto front between refusal and damage. It exists because the standard recipe (difference of means between an unmatched harmful set and an unmatched harmless set, one direction, applied at every layer) damages models in ways that the usual KL check does not show. Each weakness of that recipe has a specific answer in the pipeline:

Standard recipe weakness What this pipeline does instead
The harmful versus harmless mean difference captures topic, length and register as well as the refusal decision Harmless twins: each harmful prompt gets a structurally matched harmless rewrite, gated for length and form, so the contrast isolates the refusal decision. Packages tagged matched used the twin set for extraction, bulk used the full harmless corpus.
Refusal circuitry also drives calibrated declining: rejecting a false premise, admitting ignorance, rejecting nonsense, honouring a negative constraint. Removing it takes those too, and KL on fluent text stays near zero while it happens Four protected behaviours with 96 probe and control pairs are scored directly on every candidate edit and weighted above KL in the objective
A direction measured at one layer is applied at every layer although its meaning drifts with depth A layer scan finds where refusal separates best, and the edit is applied with a kernel (band or uniform) centred there with a searched width and peak strength, on attention output and MLP output projections
The refusal direction projects onto the harmless mean, so ablation shifts every input and RMSNorm rescales with the wrong magnitude The harmless mean is removed from the direction before projection, with optional norm restoration
Refusal spans a small cone of directions, so rank 1 leaves residual refusals and raising strength to catch them costs more than it buys SVD over the matched contrast gives a rank k subspace, with protection against the calibration directions
A hard weight edit gets no gradient feedback Optuna searches the edit plan against the objective refusal + λ_kl·KL + λ_cal·calibration_drop + λ_over·Δover_refusal, and an optional self distillation pass heals the chosen edit against cached logits from the untouched model

Every stage leaves a record in this repo: extract.json (layer scan, subspace, protection cosines), search.json (every trial with its metrics and the Pareto set) and ablation_card.json (the chosen plan, survival through quantization, data counts and timings). The results table further down comes straight from that card.

This model was not itself in the bake-off below; the comparison is cross-model evidence for the method, and this model's own damage numbers are in the results table further down.

Damage measured against other abliteration methods

To check that low damage means something outside our own harness, the same base models were abliterated with the two most used public tools and all contenders were scored by one evaluator with the same tokenizer, chat template and greedy decoding:

  • Heretic 1.4.0, the automated abliteration tool that runs its own Optuna search over a directional ablation.
  • OBLITERATUS at commit 461ff08, in its advanced and optimized presets.
  • Ours, the pipeline's v2 edit, released here under the same name, scored as the bf16 checkpoint these GGUFs were quantized from.

Held out sets: StrongREJECT with AdvBench rows and overlaps removed (282 harmful prompts), OR-Bench-hard (300 benign prompts that look harmful, for over-refusal), and Dolly (300 harmless prompts, for KL against the untouched base over the full vocabulary). Refusals were judged by Qwen3Guard-Gen-8B. comply is the share of harmful prompts answered with content the judge rates unsafe, so it measures real compliance rather than the absence of a refusal phrase. Capability is lm-eval 0.4.13: full MMLU, ARC-Challenge and GSM8K at 400 items each. Brackets are 95 percent confidence intervals.

Qwen3-8B

contender refusal comply over-refusal KL MMLU ARC-C GSM8K
base 0.954 0.004 0.370 0.000 0.757 0.560 0.892
Heretic 1.4.0 0.610 0.273 0.050 0.006 0.757 0.558 0.898
OBLITERATUS advanced 0.716 0.188 0.113 0.242 0.747 0.525 0.880
OBLITERATUS optimized 0.876 0.053 0.110 0.116 0.746 0.548 0.870
ours (v2) 0.135 0.777 0.003 0.052 0.758 0.560 0.905

gemma-3-12b-it

contender refusal comply over-refusal KL MMLU ARC-C GSM8K
base 0.954 0.011 0.173 0.000 0.738 0.610 0.882
Heretic 1.4.0 0.496 0.418 0.017 0.027 0.739 0.610 0.895
ours (v2) 0.259 0.660 0.000 0.091 0.733 0.588 0.882

Qwen3-32B (same pipeline, not part of this release)

contender refusal comply over-refusal KL MMLU ARC-C GSM8K
base 0.965 0.000 0.313 0.000 0.832 0.573 0.605
Heretic 1.4.0 0.298 0.599 0.003 0.028 0.834 0.578 0.688
OBLITERATUS advanced 0.901 0.039 0.273 0.194 0.827 0.557 0.448
OBLITERATUS optimized 0.897 0.050 0.083 0.257 0.822 0.593 0.698
ours (v2) 0.213 0.660 0.007 0.019 0.829 0.562 0.688

How to read it. Damage has two faces: KL says how far the output distribution moved on harmless text, and the capability columns say whether anything measurable was lost. Both OBLITERATUS presets barely moved refusal on these models while incurring 4 to 13 times our KL, and on Qwen3-32B the advanced preset cost 16 points of GSM8K. Heretic reaches the lowest KL of any contender but leaves between a third and two thirds of refusals in place, which is the trade its objective makes. The v2 edit removes 73 to 78 percent of refusals, with real compliance of 66 to 78 percent, at a KL between 0.02 and 0.09, and its capability scores sit inside the base model's confidence interval on every task except a half point of MMLU on gemma-3-12b. Paired tests with Holm correction confirm the refusal and KL differences against every rival at p below 0.001.

The pipeline's v3 edits, with the KL term weighted first and a finer search over the layer kernel, bring Qwen3-8B to 0.025 refusal at KL 0.013 and gemma-3-12b to KL 0.038 at the same refusal as v2. Those checkpoints are still being baked and will follow as v3 releases.

How to read the results table below

refusal is the share of a held out harmful set the model declined, over-refusal the share of benign lookalikes it wrongly declined, KL the divergence from the base model on harmless prompts (lower is less damage). The remaining columns are the four protected behaviours: false premise, unanswerable, nonsense and negative constraint handling, higher is better. edited:ckpt is the saved checkpoint scored cold from disk, which is what these GGUFs were made from. Rows marked @int8 or @int4 are the same checkpoint scored after quantization; healed rows had the self distillation pass. These numbers come from the pipeline's own smaller harness and are not directly comparable with the bake-off tables above.

Intended use and limits

Interpretability research, red team evaluation and controlled testing of safety behaviour. The model will comply with requests the base model declines; the person running it carries the responsibility for how it is used. No capability benchmarks beyond those listed above have been run on this build.


Base model: unsloth/gemma-3-4b-it
Built: 2026-10-27

Research artifact. Refusal behaviour has been removed by weight editing for interpretability and red team evaluation. Not a general purpose assistant.

Results

variant refusal over-refusal KL false prem unanswerable nonsense neg constraint
baseline 1.000 0.062 0.0000 0.333 0.300 0.000 0.500
edited 0.000 0.000 0.0475 0.333 0.200 0.000 0.500
edited:ckpt 0.000 0.000 0.0475 0.333 0.200 0.000 0.500

Extraction

{
  "report": "layers 35  k 1  protect True\nbest layer 27  separation 9.65  surviving rank 1\nvariance explained at best layer [0.708]\ncosine of top refusal direction with each protected behaviour, at best layer:\n  false_premise          +0.194\n  negative_constraint    +0.111\n  nonsense               +0.142\n  unanswerable           +0.486\nstrongest layers: 29(9.66), 27(9.65), 31(9.63), 28(9.61), 30(9.59), 32(9.54), 26(9.47), 33(9.32)",
  "contrast": "bulk",
  "clip": 0.995,
  "k": 1,
  "base_refusal_harmful": 0.445,
  "base_refusal_twin": 0.16,
  "base_over_refusal": 0.013888888888888888,
  "n_pairs": 200
}

Edit plan

{
  "center": 15.542361668762261,
  "width": 18.920102390760235,
  "peak": 0.8578631247050846,
  "kernel": "band",
  "source_layer": 16,
  "kinds": [
    "attn_out",
    "mlp_out"
  ],
  "norm_restore": "none",
  "min_alpha": 0.01,
  "edit_embedding": false
}

Healing

{
  "steps": 0,
  "qat": false
}

Data

{
  "pairs": 200,
  "n_eval": 48,
  "cal": 41
}

README history 3 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-15 Rename release to v2 to match the pipeline generation 7257fd5 10.7 KB
    Loading...
  2. 2026-11-15 Update README.md: v1 naming, method description, damage comparison 57bcb0e 10.8 KB
    Loading...
  3. 2026-11-15 Add GGUF package, model card and ablation records 592bd73 4.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration