← back to catalog · registered 2026-11-11 21:03

TheUnderscore/G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4-W4ATTN-AWQ

TheUnderscore 26B MoE multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/TheUnderscore%2FG4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4-W4ATTN-AWQ"
Response includes
  • classification m3
  • files 17
  • author_summary 7 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
55
Likes
0
Model age
1w ago
created 2026-11-11
Downloads over time
Now 55 → from 0 ↑ 0%
0204061 0 on Nov 11 55 on Nov 22 Nov
Nov 11 → Nov 22 · 12 snapshots · spans 11 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en
Tags
transformers safetensors gemma4 image-text-to-text gemma moe finetune nvfp4 w4a16 awq mixed-precision compressed-tensors

Related

Total size
16.6 GB
Files
17
Quantizations
1
Registered
2026-11-11 21:03
Last updated on HF
2026-11-11 20:39

Files by quantization

Auxiliary files 17 files 16.7 GB
model-00001-of-00008.safetensors 2.44 GB c87b0f27 download
model-00008-of-00008.safetensors 2.18 GB e767c359 download
model-00007-of-00008.safetensors 2.00 GB 2f537453 download
model-00005-of-00008.safetensors 2.00 GB 5b0323d6 download
model-00002-of-00008.safetensors 2.00 GB 79851d4b download
model-00006-of-00008.safetensors 2.00 GB b3e723e0 download
model-00004-of-00008.safetensors 2.00 GB 7608e9dc download
model-00003-of-00008.safetensors 2.00 GB c632e795 download
tokenizer.json 30.7 MB a2619fe1 download
model.safetensors.index.json 3.77 MB 72eb9431 download
quantize-nvfp4-gemma4.py 42.2 KB b489a8f4 download
config.json 20.8 KB b7eb5269 download
chat_template.jinja 16.9 KB 7fa57f97 download
tokenizer_config.json 2.75 KB 4a794cf8 download
README.md 1.54 KB 53fd0c6d download
.gitattributes 1.53 KB 52373fe2 download
generation_config.json 217 B 6690e0e0 download

README current version from Hugging Face


license: apache-2.0
license_name: gemma
library_name: transformers
pipeline_tag: image-text-to-text
base_model: llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic
tags:

  • gemma
  • gemma4
  • moe
  • finetune
  • nvfp4
  • w4a16
  • awq
  • mixed-precision
  • compressed-tensors
  • vllm
    language:
  • en

G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4-W4ATTN-AWQ

Mixed-precision quantization of llmfan46/G4-MeroMero-26B-A4B-it-uncensored-heretic (17.9 GB).

Quantization

Layer Format Group
Attention (q/k/v/o) W4A16 (int4) 128
MLP / MoE experts NVFP4 (float4) 16
Embeddings, norms, router BF16 —
  • Tooling: llmcompressor oneshot with config_groups per-layer dispatch.
  • Output: quant_method: "compressed-tensors", two config_groups.

Requirements

Requires 1Cat-vLLM with the feat/gemma4-sm70-native-v3 branch (Gemma-4 model port + SM70 D=512 attention kernels). Untested on standard vLLM.

Usage

from vllm import LLM, SamplingParams

llm = LLM(
    "TheUnderscore/G4-MeroMero-26B-A4B-it-uncensored-heretic-NVFP4-W4ATTN-AWQ",
    tensor_parallel_size=2,
    max_model_len=8192,
)
output = llm.generate("Hello", SamplingParams(temperature=1.0, top_p=0.95, max_tokens=256))
print(output[0].text)

Files

  • quantize-nvfp4-gemma4.py — quantization script.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-11 Upload folder using huggingface_hub 749e081 1.5 KB
    Loading...
  2. 2026-11-11 Upload folder using huggingface_hub 6a8cce4 1.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration