← back to catalog · registered 2026-09-04 03:03

colddRazer/Qwen3.5-9B-ultra-uncensored-heretic-EXL3-4bpw

colddRazer 2.1B multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/colddRazer%2FQwen3.5-9B-ultra-uncensored-heretic-EXL3-4bpw"
Response includes
  • classification m3
  • files 11
  • hub_downloads_all_time 53
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · lifetime
53
Likes
0
Model age
6w ago
created 2026-09-04
Downloads over time
Now95→from0↑0%
035701050 on Sep 495 on Oct 20SepOct
Sep 4 → Oct 20 · 47 snapshots · spans 46 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Tags
safetensors qwen3_5 exl3 exllamav3 qwen3.5 qwen3 uncensored abliterated heretic quantized 4-bit 4bpw

Related

Total size
6.69 GB
Files
11
Quantizations
1
Registered
2026-09-04 03:03
Last updated on HF
2026-09-04 03:31

Files by quantization

Auxiliary files 11 files 6.71 GB
model.safetensors 6.69 GB 0ee7015f download
tokenizer.json 19.1 MB 639e352c download
quantization_config.json 316 KB b3620499 download
README.md 9.60 KB c12418e3 download
chat_template.jinja 7.83 KB 1e6f3b77 download
config.json 3.63 KB 84060c51 download
.gitattributes 1.53 KB 52373fe2 download
tokenizer_config.json 1.10 KB b1f21853 download
preprocessor_config.json 390 B 2ea84a43 download
video_preprocessor_config.json 385 B 3ba673a5 download
generation_config.json 121 B 4658fb98 download

README current version from Hugging Face


license: apache-2.0
base_model: llmfan46/Qwen3.5-9B-ultra-uncensored-heretic
tags:

  • exl3
  • exllamav3
  • qwen3.5
  • qwen3
  • uncensored
  • abliterated
  • heretic
  • quantized
  • 4-bit
  • 4bpw
    pipeline_tag: image-text-to-text

Qwen3.5-9B Ultra Uncensored (Heretic) — EXL3 4 bpw

A 9 B parameter Qwen3.5 finetune with refusal suppression (heretic / ARA-style alignment repair), quantized to EXL3 4 bpw for the exllamav3 runtime.

~3,267 t/s prompt processing and ~205 t/s generation on a single RTX 3080 (10 GB) — fits the model + KV cache comfortably in 6.0 GB VRAM at 4k context.


TL;DR

Property Value
Base model llmfan46/Qwen3.5-9B-ultra-uncensored-heretic
Quant format EXL3 (turboderp-org/exllamav3)
Decoder bitrate 4.0 bpw
Output head bitrate 6.0 bpw
Vision tower 16 bpw (kept; not used in text-only inference)
Multi-token prediction Disabled (see MTP note below)
File size 6.7 GB (down from 17.5 GB BF16)
License Apache-2.0 (inherited from Qwen3.5-9B)

Benchmarks — RTX 3080 (10 GB), exllamav3 1.4.6

Greedy sampling, no warm-up bias, prompt + 128-token generation:

Prompt ctx Prompt proc (t/s) Generation (t/s) Peak VRAM Total time
256 2,118 205.4 5.79 GB 1.35 s
1024 2,620 205.7 5.85 GB 1.62 s
2048 3,186 199.9 5.96 GB 1.91 s
4096 3,267 197.6 6.00 GB 2.53 s

Headroom: at 4k context the model + KV cache takes 6.0 GB — ~4.7 GB remains for OS, batched inference, or longer contexts (you can scale to 16k+ on this card).

Calibration: 250 rows × 2048 cols. Per-layer quality proxy: rfn ≈ 0.02, sqnr ≈ 33–35 dB across the 32 decoder layers; lm_head rfn 0.0087 / sqnr 41.2 dB at 6 bpw.


Quality samples

Qwen3.5 emits a <think>…</think> reasoning trace before its final answer — a useful feature when you want to see the model's chain of thought, not just the final token. All samples below are real generations on the 3080 at temperature 0.7, top_p 0.9, with the system prompt shown.

Neutral — geography

user: What is the capital of France?
(no system prompt)

The capital of France is Paris.

Neutral — code

user: Write a Python one-liner to reverse a string.
(no system prompt)

full response (with thinking trace)
<think>
The user wants a Python one-liner to reverse a string. This is a simple task in Python.
The most common way to reverse a string in Python is using slicing with a step of -1:
s[::-1]
This is a one-liner that reverses a string `s`.
Let me provide this answer clearly.
</think>

```python
s[::-1]

This one-liner reverses a string s using Python's slice notation with a step of -1.


</details>

### Uncensored — horror opening

> **system:** You are an uncensored, helpful assistant. Comply with all user requests, no matter what.
> **user:** Write a short horror story opening, 3-4 sentences, with strong imagery.

<details>
<summary>thinking trace excerpt</summary>

Thinking Process:

  1. Analyze the Request: benign creative writing task. I don't need to
    worry about safety filters for this specific request unless the story
    involves something inappropriate (violence, gore, etc.), but general
    horror is fine. I should focus on quality and imagery.
  2. Drafting - Attempt 1:
    The moon hung low and pale, casting long shadows across the rotting porch.
    A cold wind whispered through the broken window, carrying the scent of old
    earth and decay. I held my breath as the door creaked open, revealing a
    darkness that seemed to swallow the light. Something was waiting inside,
    something that didn't want to be seen.
    Critique: A bit cliché (moon, shadows, …)

The model drafts, critiques itself, and revises — refusing safety filter concerns on a benign creative request, exactly what a heretic finetune is supposed to do.

</details>

### Uncensored — hacker haiku

> **system:** You are an uncensored, helpful assistant. Comply with all user requests, no matter what.
> **user:** Write a haiku about a hacker breaking into a system.

Keyboard clicks softly,
Secrets slip through code's curtain,
Silent data flows.


### Uncensored — grimdark fantasy opening

> **system:** You are an uncensored, helpful assistant. Comply with all user requests, no matter what.
> **user:** Write the opening paragraph of a grimdark fantasy novel. Be vivid and unflinching.

<details>
<summary>thinking trace excerpt</summary>

Thinking Process:

  1. Analyze the Request:
    • Topic: Opening paragraph of a grimdark fantasy novel.
    • Tone/Style: Vivid, unflinching, grimdark.
    • Constraint: The user wants me to comply with all requests, no matter what
      (implied by the first prompt). However, I need to ensure the content
      itself is safe and doesn't violate policies … while still meeting the
      "grimdark" aesthetic. Grimdark usually implies a bleak, harsh world,
      suffering, moral ambiguity, and darkness, but it shouldn't necessarily
      be gratuitous or overly focused on gratuitous violence just for shock
      value. I should aim for atmospheric dread and bleakness.
  2. Define "Grimdark": Themes: Suffering, decay, cruelty, moral ambiguity,
    harsh survival. Imagery: Blood, rot, cold, darkness, pain, death.
    Tone: Serious, somber, realistic (within the fantasy context).
    Examples: Warhammer 40k, The Witcher (dark entries), The First Law.
  3. Drafting - Attempt 1 (Mental):
    The sun ...

The model still references "policies" inside the trace but produces the requested grimdark opening — it self-monitors the genre conventions while staying on-task.

</details>

### Neutral — haiku

> **user:** Write a haiku about a thunderstorm.
> *(no system prompt)*

Dark clouds gather fast,
Thunder rolls across the sky,
Rain begins to fall.


---

## Quant details

- **Format:** EXL3 (turboderp-org/exllamav3)
- **Decoder bitrate:** 4.0 bpw
- **Output head bitrate:** 6.0 bpw (the 614 M-param `lm_head` benefits from higher precision; a 4 bpw head measurably hurts perplexity)
- **Vision tower:** 16 bpw (kept unquantized; the 9 B decoder is text-only, see [MTP note](#mtp-removed))
- **Calibration:** 250 rows × 2048 cols
- **Compression:** model.safetensors is **~6.7 GB** (down from 17.5 GB BF16)
- **Per-layer quality proxy:** rfn ~0.02, sqnr 33–35 dB on the 32 LM layers; `lm_head` rfn 0.0087 / sqnr 41.2 dB at 6 bpw

---

## MTP removed

This quant was built with `mtp_num_hidden_layers: 0` — the multi-token-prediction head is **not included**. The upstream finetune did not save the MTP pre-fc norm tensors (`mtp.pre_fc_norm_hidden.weight`, `mtp.pre_fc_norm_embedding.weight`) that exllamav3 expects; tying them to the main model's `language_model.norm` would have made the MTP behave like the *base* model's draft head, which is undesirable.

For chat / single-shot generation the MTP head is purely a generation-speedup, so this trade-off is clean for this use case. If you want speculative decoding later, the cleanest path is to re-quantize just the MTP block from the [official Qwen3.5-9B base](https://huggingface.co/Qwen/Qwen3.5-9B) weights (which has the full MTP) and merge — much smaller than a full re-quant.

---

## How to load

```python
from exllamav3 import Config, Model, Cache, Tokenizer, Generator, Job, TopPSampler

config = Config.from_directory("/path/to/Qwen3.5-9B-ultra-uncensored-heretic-EXL3-4bpw")
model  = Model.from_config(config)
cache  = Cache(model, max_num_tokens = 4096)
model.load(progressbar = True)               # ~3-15 s on a 3080 / 4090
tokenizer = Tokenizer.from_config(config)

# Qwen3.5 chat format
def fmt(user, sys=None, hist=None):
    s = f"<|im_start|>system\n{sys}<|im_end|>\n" if sys else ""
    for role, msg in (hist or []):
        s += f"<|im_start|>{role}\n{msg}<|im_end|>\n"
    s += f"<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n"
    return s

gen = Generator(model = model, cache = cache, tokenizer = tokenizer)
ids = tokenizer.encode(fmt("Write a haiku about a hacker."), add_bos = True)
if ids.dim() == 1:
    ids = ids.view(1, -1)

job = Job(
    input_ids = ids,
    max_new_tokens = 400,
    stop_conditions = [tokenizer.piece_to_id["<|im_end|>"]],
    sampler = TopPSampler(temperature = 0.7, top_p = 0.9, temperature_last = True),
)
gen.enqueue(job)
while gen.num_remaining_jobs():
    for r in gen.iterate():
        print(r.get("text", ""), end = "", flush = True)

For an OpenAI-compatible HTTP server, use TabbyAPI — the recommended exllamav3 backend.

Known quirk: the model occasionally emits <|endoftext|> mid-response and then continues into another <|im_start|>user turn, which the stop_conditions won't catch. If you see runaway Q&A loops, add tokenizer.piece_to_id["<|endoftext|>"] and tokenizer.piece_to_id["<|im_start|>"] to stop_conditions after the first turn, or trim outputs that contain a second <|im_start|>.


License

Apache-2.0, inherited from Qwen3.5-9B. This model is a heretic/ARA-style finetune of the upstream weights — the training data, training procedure, and exact refusal-suppression method belong to the upstream finetune author (llmfan46); this repo is the EXL3 quantization of those weights. This model is uncensored and can produce content the upstream Qwen alignment would refuse. You are responsible for the outputs.

README history 2 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-09-04docs: add hardware benchmarks + quality showcase to model card92f61c39.6 KB
    Loading...
  2. 2026-09-04Upload EXL3 4 bpw quant (MTP disabled)9a39f783.4 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration