license: apache-2.0
license_link: LICENSE
base_model: lukey03/Qwen3.5-9B-abliterated
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags:
- qwen
- qwen3.5
- gguf
- llama.cpp
- abliterated
- uncensored
Qwen3.5-9B-abliterated-GGUF
This repo holds one GGUF file of lukey03/Qwen3.5-9B-abliterated. That model is an abliterated (refusal-removed) build of Qwen/Qwen3.5-9B.
The file is a byte-for-byte copy of lukey03's quant in lukey03/Qwen3.5-9B-abliterated-GGUF. It has not been requantized or modified. It is republished here, under the Apache-2.0 license, as the Bronze tier of DeepSeek Unchained. All credit goes to the Qwen team for the model, and to lukey03 for the abliteration and quantization.
Disclaimer
This model has had its safety alignment substantially removed. It will comply with requests that the original Qwen3.5-9B refuses. You are responsible for how you use it and for what it generates. Use it within the Apache-2.0 license and applicable law. Abliteration removes refusals; it does not add knowledge or remove biases from the base model's training. lukey03's model card has more detail.
Files
| Path | Quant | Size (bytes) | sha256 |
|---|---|---|---|
Qwen3.5-9B-abliterated-Q4_K_M.gguf |
Q4_K_M | 5,627,044,704 | dba64d0e5cce0739e27535ee0a6b75249eb8006ce8b2d6c060e20750035c4695 |
The hash matches the file in the lukey03 repo. This is the text-only file, with no mmproj. It needs a recent llama.cpp with the qwen35 architecture, and it was tested with b10919. Qwen3.5 thinks by default; pass enable_thinking=false through the chat template (llama.cpp: --reasoning off) to turn it off.
How the quant was chosen
lukey03's Q4_K_M and four quants made from lukey03's F16 with llama.cpp b10919 llama-quantize (no imatrix) were measured against that F16, on one Apple M4 Max with 128 GB of memory. KL divergence against the F16 logits comes from llama-perplexity (wikitext-2-raw test, ctx 512, 100 chunks). Speed comes from llama-bench.
| Quant | File GB | Mean KLD | p99 KLD | Same top token % | PPL ratio vs F16 | pp512 t/s | tg128 t/s |
|---|---|---|---|---|---|---|---|
| Q3_K_M | 4.62 | 0.1757 | 1.371 | 81.15 | 1.0230 | 699.6 | 46.57 |
| Q4_K_M | 5.63 | 0.0530 | 0.455 | 88.86 | 1.0352 | 739.2 | 50.54 |
| Q5_K_M | 6.47 | 0.0170 | 0.147 | 94.13 | 1.0165 | 728.9 | 44.54 |
| Q6_K | 7.36 | 0.0055 | 0.043 | 96.57 | 1.0012 | 763.3 | 45.50 |
| Q8_0 | 9.53 | 0.0005 | 0.004 | 98.94 | 1.0018 | 882.2 | 41.73 |
Q4_K_M keeps the tier close to the size of the model it replaces. Q3_K_M loses much more fidelity. Lower KLD means the quant's output distribution is closer to F16. "Same top token" is how often the quant and F16 agree on the most likely next token. Only the Q4_K_M file is published here.
License
Apache-2.0, inherited from Qwen/Qwen3.5-9B. The LICENSE file is copied from that repo.