license: agpl-3.0
language:
- en
base_model: - helixdouble/GLM-5.1-Abliterated
base_model_relation: quantized
library_name: llama.cpp
pipeline_tag: text-generation
tags: - gguf
- q5_k_m
- glm
- glm-5.1
- glm-dsa
- moe
- quantized
- abliterated
- llama.cpp
GLM-5.1-Abliterated Q5_K_M GGUF
TL;DR
- For agentic / long-horizon / coding workloads where you don't want refusals on technical questions
- 754B MoE (40B active), abliterated, Q5_K_M, 534 GB / 33 shards
- ~37 t/s decode for ~$8-10/hr on 8× RTX PRO 6000 Blackwell (vast.ai)
- ~23 t/s decode for ~$8-10/hr on 8× A100-SXM4 (vast.ai)
- Full 200K context tested, with
q8_0KV cache - AIME 2026: 6/6 correct on the first six problems (I did partial run with thinking-on)
- 8× 80GB or 8× 96GB GPU for the recommended config
GGUF Q5_K_M quantization of helixdouble/GLM-5.1-Abliterated. Runs in mainline llama.cpp, LM Studio, Ollama.
Abliterated means the model was modified to reduce refusal behavior. It removes the directions in the model's activation space that produce refusals, without retraining. The result is a model that declines less often on prompts it would otherwise refuse, while keeping general capability intact. This quant inherits that behavior from upstream — no additional abliteration was applied here.
The model will engage with technical questions that mainstream chat models often over-refuse (security research, defensive tooling, dual-use). However, this model still declines to provide methods on self-harm queries and similar.
Performance
--parallel 1, q8_0 KV cache, flash attention on.
8× RTX PRO 6000 Blackwell 96GB
ctx=202752
| Test | Prefill t/s | Decode t/s |
|---|---|---|
| 256-token gen, short prompt | 215 | 36.80 |
| 8k prompt + 128 gen | 659 | 34.05 |
| 32k prompt + 128 gen | 502 | 28.57 |
| 128k prompt + 128 gen | 240 | 17.91 |
| short prompt + 2k gen | 295 | 36.14 |
~$8-10/hr on vast.ai. huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated or similar model also fits in VRAM simultaneously - you can use it for subagent tasks!
8× A100-SXM4 80GB
ctx=202752
| Test | Prefill t/s | Decode t/s |
|---|---|---|
| short prompt, 256 gen | 55 | 23.27 |
| short prompt, 512 gen | 59 | 23.00 |
| short prompt, 1024 gen | 57 | 22.83 |
| 4.5k prompt + 128 gen | 209 | 22.43 |
~$8-10/hr on vast.ai.
2× RTX PRO 6000 Blackwell 96GB
ctx=8192, mostly offloaded to RAM
| Run | Output tokens | Prefill t/s | Decode t/s |
|---|---|---|---|
| prompt 60 tokens, 256 gen | 256 | 18.56 | 5.43 |
| prompt 67 tokens, 512 gen | 512 | 19.65 | 5.33 |
| prompt 2222 tokens, 128 gen | 66 | 81.48 | 5.37 |
| prompt 45 tokens, no max_tokens | 1976 | 14.98 | 5.47 |
~$3-5/hr on vast.ai.
Full-config numbers above are the recommended hardware target. Two-GPU operation works but the model does not fully fit in VRAM at this configuration.
Quality — AIME 2026
Just to quickly check nothing went wrong, I ran partial evaluation of AIME 2026.
1 attempt per problem, sampling temperature=1.0 top_p=0.95, enable_thinking=true, default matharena evaluator.
| Problem | Wall time | Reasoning tokens | Result |
|---|---|---|---|
| 1 | 0m 48s | 1,089 | ✓ |
| 2 | 6m 08s | 8,117 | ✓ |
| 3 | 5m 16s | 6,987 | ✓ |
| 4 | 9m 05s | 11,869 | ✓ |
| 5 | 2m 19s | 3,131 | ✓ |
| 6 | 1m 29s | 2,005 | ✓ |
6/6 correct on the first 6 problems consecutively.
Quick start
- Start llama-server and let it download the GGUF from Hugging Face:
llama-server \
--hf-repo arxorry/GLM-5.1-Abliterated-Q5_K_M-GGUF \
--hf-file GLM-5.1-Abliterated-Q5_K_M-00001-of-00033.gguf \
--ctx-size 202752 \
--flash-attn on \
--cache-type-k q8_0 \
--cache-type-v q8_0
Open the llama.cpp web UI or connect with an OpenAI-compatible client.
For manual downloads, keep all 33 shards in the same directory and load the first shard.
Prompt Format
GLM-5.1 uses its own chat template with explicit <think> / </think> blocks for the reasoning trace. The full jinja template is embedded in the GGUF.
Token markers:
[gMASK]<sop><|system|>system prompt<|user|>user message<|assistant|><think>reasoning</think>visible answer
Thinking mode is on by default. To disable, pass enable_thinking=false in chat_template_kwargs:
{
"messages": [{"role": "user", "content": "Hello"}],
"chat_template_kwargs": {"enable_thinking": false}
}
OpenAI-compatible clients work directly with the /v1/chat/completions endpoint.
Source
This was quantized from helixdouble/GLM-5.1-Abliterated, which is based on zai-org/GLM-5.1-FP8 / zai-org/GLM-5.1. This release changes the storage/runtime format and quantization only.
Quantization Recipe
Direct Q5_K_M from BF16 via llama-quantize. No imatrix calibration, no per-tensor overrides. Reproducible end-to-end from helixdouble/GLM-5.1-Abliterated FP8 source with a single command.
Conversion timings (CPU-only, 128 threads, ~3 TB scratch disk):
- HF safetensors -> BF16 GGUF: 55m 14s
- BF16 GGUF -> Q5_K_M GGUF: 31m 37s
Notes
- Benchmarks above are single-client. I haven't run concurrent-client throughput.
- For production deployment add your own guardrails - abliterated != aligned.
Feedback
Open an issue in the Community tab. This is my first quant, feedback is genuinely useful😀
Support
If this quant saved you some vast.ai bucks, a tip helps fund the next one.
Disclaimer
Provided AS IS for research and educational purposes. This model has reduced refusal behavior inherited from upstream - outputs may be inaccurate, biased, unsafe, or that you find may offensive. You are responsible for compliance with applicable laws in your jurisdiction and for any guardrails you add when deploying.
No warranty is given, no liability accepted for downstream use.
License
This GGUF follows the source model licensing. The source model is listed as AGPL-3.0 and also refers users to the upstream GLM-5.1-FP8 license. For redistribution, modification, or hosted use, check the upstream model cards and license files.