base_model: alexander2323/Qwen3.8-27B-Uncensored-Heretic-Abliterated
tags:
- exl3
- quantized
- qwen3
- abliterated
- uncensored
Qwen3.8-27B-Uncensored-Heretic-Abliterated — EXL3 Quantizations
This repository contains EXL3 quantizations of alexander2323/Qwen3.8-27B-Uncensored-Heretic-Abliterated at multiple bitrates. Each quantization lives on its own branch.
Quick Links
| Branch | Bitrate | Head | Shards | Size (approx.) |
|---|---|---|---|---|
2.00bpw |
2.00 bpw | 4 bpw | 2 | ~13 GB |
3.00bpw |
3.00 bpw | 4 bpw | 2 | ~20 GB |
4.00bpw |
4.00 bpw | 6 bpw | 2 | ~27 GB |
5.00bpw |
5.00 bpw | 6 bpw | 3 | ~33 GB |
6.00bpw |
6.00 bpw | 6 bpw | 3 | ~40 GB |
8.00bpw |
8.00 bpw | 8 bpw | 4 | ~53 GB |
Quantization Details
All quantizations were generated with exllamav3 using:
- Calibration: 250 rows × 2048 tokens
- Codebook: mul1
- Out scales: always
- Hardware: 2× RTX 5090 (32 GB) + 1× RTX 6000 Blackwell (96 GB)
- Device ratios: 2:2:4 (GPU 0:1:2)
Usage
# Clone a specific quantization branch
git clone -b 4.00bpw https://huggingface.co/genevera/Qwen3.8-27B-Uncensored-Heretic-Abliterated-exl3
# Or with git-lfs
git lfs install
git clone https://huggingface.co/genevera/Qwen3.8-27B-Uncensored-Heretic-Abliterated-exl3
cd Qwen3.8-27B-Uncensored-Heretic-Abliterated-exl3
git checkout 4.00bpw
Load with exllamav3:
from exllamav3 import ExLlamaV3
from exllamav3.config import ExLlamaV3Config
config = ExLlamaV3Config()
config.model_dir = "./"
model = ExLlamaV3(config)
Source Model
alexander2323/Qwen3.8-27B-Uncensored-Heretic-Abliterated — an abliterated + uncensored variant of Qwen3 27B.
License
Same as the source model. See alexander2323/Qwen3.8-27B-Uncensored-Heretic-Abliterated for details.