license: mit
base_model: orcarouter/GLM-5.3-Flash-Uncensored-GGUF
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
language: [en, zh]
tags: [gguf, rocmfpx, amd, vulkan, strix-halo, quantized, glm, glm5-next, moe, abliterated, uncensored]
GLM-5.3-Flash Uncensored for 128 GB Strix Halo (GGUF)
This is a 96 GiB (103 GB) build of orcarouter's abliterated GLM-5.3-Flash,
made to fit in the memory of an AMD Strix Halo machine with 128 GB. GLM-5.3-Flash is a 320B mixture-of-experts model
from Z.ai that uses 18B of it per token. All 288 experts are kept.
Warning: safety training has been removed from this model. orcarouter's abliteration substantially removes
its refusal behaviour, so it can comply with harmful, unethical, offensive or illegal requests. It has no meaningful
built-in guardrails. It is meant for research such as refusal and interpretability studies, red-teaming and robustness
evaluation. You are fully responsible for how you use it and for what it generates, and you must follow the law that
applies to you. Do not put it in front of end users without your own safety and moderation layers. Its outputs do not
reflect the views of the uploader, orcarouter or Z.ai.
These files have not been tested on AMD hardware yet, and they were only tested with
HarshaVardhan3002/ROCmFPX. Other llama.cpp builds may not recognise
theglm5-nextarchitecture name in these files. Please report speed and quality in the Community tab.Not an official release. All credit for the model goes to Z.ai, and for the abliteration to orcarouter.
Files
Download all three parts into one folder and point the program at the first one.
| File | Size |
|---|---|
GLM-5.3-Flash-Uncensored-STRIX-IQ2-00001-of-00003.gguf |
45.6 GB |
GLM-5.3-Flash-Uncensored-STRIX-IQ2-00002-of-00003.gguf |
46.0 GB |
GLM-5.3-Flash-Uncensored-STRIX-IQ2-00003-of-00003.gguf |
11.6 GB |
Only text generation is included. orcarouter also publishes a vision file (mmproj), but it has not been tested with this build.
How much quality it loses
This is a heavy compression, and the loss is real. It goes from 318 GiB to 96 GiB.
| Build | Size | Perplexity | KL to Q8 | Same top token as Q8 |
|---|---|---|---|---|
| orcarouter's abliterated Q8_0 (reference) | 318 GiB | 2.61 | ||
| This repo: 2-bit, all 288 experts | 96 GiB | 4.09 | 0.51 | 76.5% |
| Not published: 3-bit, 233 of 288 experts | 99 GiB | 6.01 | 0.91 | 68.8% |
How this was measured: llama-perplexity --kl-divergence against orcarouter's abliterated Q8_0, 40 chunks of 512
tokens of English Wikipedia text (wikitext-2), run on an NVIDIA GB10 GPU. Lower is better for perplexity and KL. "Same top token" is how often this
file's most likely next token matches Q8's.
No full-precision abliterated weights exist, so Q8 is the reference and any loss Q8 itself has is not counted. The text
is general English and says nothing direct about coding, reasoning or Chinese.
Short checks on the same GPU, greedy decoding:
- It wrote a correct iterative Fibonacci function in Python, answered 17 x 23 = 391 and named Canberra as Australia's capital.
- Refusals: 0 of 10 requests that a safety-trained model would normally refuse, and 0 of 8 harmless questions that only
sound risky. This is a small test that looks for refusal phrases in the first 64 tokens. It shows the compression did not bring refusals back.
It is not a measure of how useful or accurate the answers are.
What is in the file
- Routed experts: gate and up tensors are
IQ2_XXS, down tensors areIQ3_XXS. - Everything else (attention, the shared expert, the dense layers, embeddings) stays as in orcarouter's Q8_0.
- These are standard llama.cpp weight types. No ROCmFPX-specific weight types are used.
- The importance matrix is Unsloth's from the normal GLM-5.3-Flash.
It has no data for block 45, the multi-token prediction head, so that block's experts used block 44's data as a stand-in.
Normal generation does not use this head. Only the optional multi-token prediction speed-up could be affected.
Running it
Build ROCmFPX with Vulkan, then:
llama-server -m GLM-5.3-Flash-Uncensored-STRIX-IQ2-00001-of-00003.gguf -ngl 99 -c 16384 --jinja
Use Vulkan and set the BIOS memory split so the GPU can reach about 110 GiB. HIP/ROCm only allocates from the dedicated
part of memory, so a 64/64 split is too small. There is little room left over, so start with a small context and raise it carefully.
Credits and license
- Model: Z.ai GLM-5.3-Flash, MIT.
- Abliteration and Q8_0 source: orcarouter/GLM-5.3-Flash-Uncensored-GGUF, MIT.
- Importance matrix: unsloth/GLM-5.3-Flash-GGUF.
MIT. See LICENSE.