← back to catalog · registered 2026-08-22 13:56

cgus/granite-3.1-8b-instruct-abliterated-exl2

Name: cgus/granite-3.1-8b-instruct-abliterated-exl2
Author: cgus

Your rig guess connected

? Why do I need an app?

Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/cgus%2Fgranite-3.1-8b-instruct-abliterated-exl2"

Response includes

classification m1
files 11
hub_downloads_all_time 151
author_summary 13 models
readme_text full

10 credits · hourly refresh · ~4 KB payload Get an API key →

Abliteration classifier · v1.0.0

Primary method

Direct removal

No other method signals detected in this model.

Confidence

MEDIUM

Why this label 3 signals

Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.

'abliterated' in name/tags
is_gguf=0 (base model)
no specific method indicators - defaulting to M1 (most common)

Refusal direction extracted via

Extraction technique

Difference-of-means

FromArditi et al. (2024) ↗

Confidence

MEDIUM

Why we say so

primary_method=M1; difference-of-means is the reference extraction for M1/M3 (Arditi 2024)

Downloads · lifetime

151

20 last 30d - stable

Likes

Model age

20mo ago

created 2025-02-03

Downloads over time

Jan 29, 2025 → Oct 12 · 129 snapshots · spans 621 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

huihui-ai/granite-3.1-8b-instruct-abliterated this lineage

cgus/granite-3.1-8b-instruct-abliterated-exl2↓ 20

Metadata

License

apache-2.0

Files by quantization

Auxiliary files 11 files 4.24 GB

output.safetensors	4.23 GB	fc99b0a2	download
tokenizer.json	3.31 MB	11f3f2c6	download
vocab.json	759 KB	0a11f201	download
merges.txt	431 KB	f8479fb6	download
tokenizer_config.json	8.08 KB	156644f7	download
README.md	2.00 KB	e6cf0aab	download
.gitattributes	1.48 KB	a6344aac	download
config.json	1.13 KB	d4fd8e94	download
special_tokens_map.json	736 B	6811616c	download
generation_config.json	139 B	e3b0c04b	download
added_tokens.json	92.0 B	3af03016	download

README current version from Hugging Face

pipeline_tag: text-generation
inference: false
license: apache-2.0
library_name: transformers
tags:

language
granite-3.1
abliterated
uncensored
base_model:
huihui-ai/granite-3.1-8b-instruct-abliterated

granite-3.1-8b-instruct-abliterated-exl2

Model: granite-3.1-8b-instruct-abliterated
Made by: huihui-ai
Granite 3 authors: Granite Team, IBM

Quants

4bpw h6 (main)
4.5bpw h6
5bpw h6
6bpw h6
8bpw h8

Quantization notes

Made with exllamav2 0.2.7 with default dataset. This model requires exllamav2 0.2.7 or newer.
Exl2 quants require to be fully loaded into GPU VRAM, RAM offloading isn't supported natively.
Additionally it requires Nvidia RTX on Windows or Nvidia RTX/AMD ROCm on Linux.
These quants can be used with TabbyAPI or Text-Generation-WebUI.

Original model card

huihui-ai/granite-3.1-8b-instruct-abliterated

This is an uncensored version of ibm-granite/granite-3.1-8b-instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

Use with ollama

You can use huihui_ai/granite3.1-dense-abliterated directly,

ollama run huihui_ai/granite3.1-dense-abliterated

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

2025-02-03Update README.md300207f2 KB
Loading...
2025-02-03Upload 10 filesdce1f6b872 B
Loading...