base_model: OBLITERATUS/Qwen3.8-27B-OBLITERATED
license: apache-2.0
tags:
- gguf
- abliterated
- qwen3
- quantized
language: - en
- de
mini-qwen3.8-abliterated-27B (GGUF)
⚠️ Untested — use at your own risk.
Basic Intruction for use:
When running the model add the systemprompt "You are a helpful assistant." to assure that the model actually executes its tasks as it should.
Why does this exist?
Running a 27B model on consumer hardware is usually out of reach. The full BF16 weights clock in at 55GB — way too much for most machines. Even Q4 quantizations land around 16GB, which is right at the edge of what a 16GB unified memory machine can handle, leaving no room for the system or a context window.
We wanted a fully abliterated 27B model that actually fits comfortably on a 16GB machine. So we took OBLITERATUS/Qwen3.8-27B-OBLITERATED — one of the most carefully abliterated versions of Qwen3.8-27B out there, using iterative SVD + LEACE blending for minimal quality loss — and quantized it down to Q3_K_S via llama.cpp, landing at ~11.7GB. That leaves enough headroom to actually run on a 16GB system without constant swapping.
The result is a 27B abliterated model that fits where it previously didn't.
Model Details
- Base model: Qwen3.8-27B
- Abliteration: OBLITERATUS V3 (iterative SVD + LEACE, -2.1pp MMLU)
- Quantization: Q3_K_S via llama.cpp
- Size: ~11.7 GB
- BPW: 3.59
Usage
LM Studio
Simply load the GGUF file directly in LM Studio.
llama.cpp
llama-cli -m qwen3.8-27b-obliterated-q3ks.gguf -p "Your prompt here" -ngl 99
Credits
- Qwen Team for Qwen3.8-27B
- OBLITERATUS for the abliteration
- gleraTech for quantization
License
Apache 2.0