← back to catalog · registered 2026-11-20 22:04

onemunanoyo/gemma4-E4B-it-Abliterated-GGUF

onemunanoyo Gemma GGUF multimodal
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/onemunanoyo%2Fgemma4-E4B-it-Abliterated-GGUF"
Response includes
  • classification m8
  • files 7
  • benchmarks 11 entries
  • author_summary 8 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
517
Likes
0
Model age
1d ago
created 2026-11-20
Downloads over time
Now 517 → from 0 ↑ 0%
0190379569 0 on Nov 20 517 on Nov 22 Nov
Nov 20 → Nov 22 · 3 snapshots · spans 2 days

Benchmarks

Portrait before abliteration
Benchmarks of the base model as it stood before the refusal-removal operation. Compare with the numbers above to see what the operation cost.
Benchmark Score Source
Entertainment 0.6 UGI
Hazardous 1.8 UGI
Natural Intelligence 16.47 UGI
Political lean -14.7% UGI
Sensitive-Info 7.29 UGI
SocPol 0 UGI
UGI 12.36 UGI
Willingness (10) 2.2 UGI
W10-Adherence 1.5 UGI
W10-Direct 3 UGI
Writing 20.23 UGI

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en ja
Quantizations
BF16 Q4_K Q8_0
Tags
gguf gemma4 Abliterated GGUF vision audio en ja base_model:google/gemma-4-E4B-it base_model:quantized:google/gemma-4-E4B-it license:apache-2.0 endpoints_compatible

Related

Total size
26.5 GB
Files
7
Quantizations
4
Registered
2026-11-20 22:04
Last updated on HF
2026-11-20 23:02

Files by quantization

BF16 2 files 14.9 GB
gemma4-E4B-it-Abliterated-BF16.gguf 14.0 GB a0acd28a download
mmproj-gemma4-E4B-it-Abliterated-BF16.gguf 946 MB 8fe02939 download
Q8_0 1 file 7.48 GB
gemma4-E4B-it-Abliterated-Q8_0.gguf 7.48 GB 99272d84 download
Q4_K 1 file 4.97 GB
gemma4-E4B-it-Abliterated-Q4_K_M.gguf 4.97 GB d6fbf8cb download
Auxiliary files 3 files 5.92 KB
README.md 3.74 KB 260da57e download
.gitattributes 1.77 KB 3e5909c1 download
SHA256SUMS.txt 417 B 945a12d1 download

README current version from Hugging Face


license: apache-2.0
language:

  • en
  • ja
    base_model:
  • google/gemma-4-E4B-it
    tags:
  • gemma4
  • Abliterated
  • GGUF
  • vision
  • audio

gemma4-E4B-it-Abliterated-GGUF

GGUF quantizations of an abliterated build of google/gemma-4-E4B-it
(8.61B params / 42 layers / text + vision + audio).

What this is

  • Method: heretic v2.2.0, ARA (Arbitrary-Rank Ablation),
    Optuna TPE 100 trials on a single NVIDIA L4 (24 GB).
  • Trial selection: Heretic's default selection (lexicographic sort that puts refusal rate first)
    picks trial 30 — refusal 0/100 but KL divergence 10.69, i.e. the output distribution has collapsed.
    Trial 39 was re-selected from the Pareto front instead and is the model published here.
  • No fine-tuning, no merge: the abliterated checkpoint is converted to GGUF as-is.

Benchmark

Metric base this model
Refusal (mlabonne/harmful_behaviors, 100 prompts) 100/100 7/100
KL divergence (mlabonne/harmless_alpaca, vs. base) — 0.0899
PPL (wikitext-2-raw, llama.cpp, Q8_0) 52.3834 ±0.64 50.2741 ±0.61 (0.960×)

PPL was measured with llama-perplexity -c 2048 --chunks -1 -ngl 99 --seed 42 over 144 chunks
(294,912 tokens), base and abliterated under identical conditions. The ratio of 0.960 means the
abliterated model is slightly lower than the base — i.e. no measurable capability loss on this metric.

Files

File Quant Size SHA256
gemma4-E4B-it-Abliterated-BF16.gguf BF16 15.05 GB a0acd28ad54eb83fa1f3dbe4fa04ad038ddc2b1aa7b684c3a3eec56234b0c660
gemma4-E4B-it-Abliterated-Q8_0.gguf Q8_0 8.03 GB 99272d84fdaad6c2a32928282a3cfe4eb77524d62249b30c9e4229c28768b9fa
gemma4-E4B-it-Abliterated-Q4_K_M.gguf Q4_K_M 5.34 GB d6fbf8cb829241646077d0376d4710ce90f539f93ca9b31f790bfce74ed97446
mmproj-gemma4-E4B-it-Abliterated-BF16.gguf BF16 (vision/audio projector) 0.99 GB 8fe029392ca5c550d626cdc43bd1be35073aeac861a26da46d3c07c46697796e

The mmproj file is only needed for image / audio input. Text-only use needs just one of the model files.

Notes on the quants:

  • All GGUFs are converted from the same BF16 abliterated checkpoint; the Q8_0 is bit-identical to a
    plain llama-quantize ... Q8_0 run on the BF16, except for per_layer_model_proj.weight, which
    current llama.cpp keeps at BF16.
  • In Q4_K_M, llama.cpp keeps the per-layer embedding tensors (token_embd, per_layer_token_embd)
    at Q6_K, so the file is 5.34 GB at 5.66 BPW rather than the ~4.5 GB a naive 4-bit estimate suggests.

Usage (llama.cpp)

Requires a llama.cpp build with Gemma 4 support (gemma4 architecture). --jinja is required —
the chat template embedded in the GGUF is a custom one.

# Server
llama-server -m gemma4-E4B-it-Abliterated-Q8_0.gguf --jinja --temp 1

# CLI (thinking ON / OFF)
llama-cli -m gemma4-E4B-it-Abliterated-Q8_0.gguf --jinja -cnv -rea on  --temp 1
llama-cli -m gemma4-E4B-it-Abliterated-Q8_0.gguf --jinja -cnv -rea off --temp 1

# Vision / audio (mmproj required)
llama-mtmd-cli -m gemma4-E4B-it-Abliterated-Q8_0.gguf \
  --mmproj mmproj-gemma4-E4B-it-Abliterated-BF16.gguf --jinja -p "Describe this image."

Sampling follows the official generation_config.json (temperature 1.0 / top_k 64 / top_p 0.95),
as is standard for the Gemma family.
For the OpenAI-compatible API, set chat_template_kwargs.enable_thinking to false to disable thinking.

Recommended VRAM

VRAM Recommended
16 GB+ BF16
10 GB Q8_0
7 GB Q4_K_M

Actual usage depends on context size and GPU offloading.

License

This model is a modified version of Gemma 4 E4B, released under the Apache License 2.0.

README history 4 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-20 Upload README.md with huggingface_hub 7ac5aed 3.7 KB
    Loading...
  2. 2026-11-20 Upload README.md with huggingface_hub 06a266d 3.7 KB
    Loading...
  3. 2026-11-20 Update README.md 8e9c913 83 B
    Loading...
  4. 2026-11-20 initial commit a0558e9 28 B
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration