library_name: llama.cpp
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.5-122B-A10B/blob/main/LICENSE
pipeline_tag: text-generation
base_model:
- wangzhang/Qwen3.5-122B-A10B-abliterated
tags: - abliterix
- uncensored
- decensored
- abliterated
- moe
- gguf
Qwen3.5-122B-A10B-abliterated-GGUF
GGUF quantized versions of wangzhang/Qwen3.5-122B-A10B-abliterated, a decensored Qwen/Qwen3.5-122B-A10B created using Abliterix.
Available Quantizations
| File | Quantization | Size | Quality | Use Case |
|---|---|---|---|---|
| Q4_K_M | Q4_K_M | 70 GB | High | 1x 80GB GPU or CPU with 96GB+ RAM |
| Q8_0 | Q8_0 | 121 GB | Very High | 2x 80GB GPUs or CPU with 160GB+ RAM |
About the Source Model
- 95% refusal reduction: Reduced from 100/100 refusals to just 5/100 (5%)
- MoE architecture: Qwen3.5-122B-A10B activates only ~10B parameters per token — 14B-class speed with 122B-class knowledge
- Minimal capability loss: KL divergence of just 0.0878 from the original model
- 50-trial Optuna TPE optimization: Automated Bayesian hyperparameter search with multi-objective Pareto optimization
- Orthogonalized abliteration: Surgical removal of refusal directions without degrading general intelligence
See wangzhang/Qwen3.5-122B-A10B-abliterated for full details on the abliteration process and steering parameters.
Usage
llama.cpp
# Q4_K_M (recommended for single GPU)
./llama-cli -m Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf -p "Your prompt here" -n 512
# Q8_0 (higher quality)
./llama-cli -m Qwen3.5-122B-A10B-abliterated-Q8_0.gguf -p "Your prompt here" -n 512
llama-server
./llama-server -m Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf --host 0.0.0.0 --port 8080
Ollama
# Create a Modelfile
echo "FROM ./Qwen3.5-122B-A10B-abliterated-Q4_K_M.gguf" > Modelfile
ollama create qwen3.5-122b-abliterated -f Modelfile
ollama run qwen3.5-122b-abliterated
VRAM / RAM Requirements
| Quantization | Full GPU Offload | Partial Offload (32 layers) | CPU Only |
|---|---|---|---|
| Q4_K_M | ~74 GB (1x 80GB) | ~40 GB GPU + 40 GB RAM | ~80 GB RAM |
| Q8_0 | ~130 GB (2x 80GB) | ~70 GB GPU + 70 GB RAM | ~140 GB RAM |
Disclaimer
This model is provided for research purposes only. The creator is not responsible for any misuse.
Credits
- Base model: Qwen/Qwen3.5-122B-A10B by Alibaba Qwen team
- Abliteration: wangzhang/Qwen3.5-122B-A10B-abliterated
- Abliteration framework: Abliterix
- Quantization: llama.cpp