license: mit
base_model:
- zai-org/GLM-5.3-Flash
- Blackfrost-Research/GLM-5.3-Flash-DERISKED-BF16
base_model_relation: quantized
tags: - omlx
- mlx
- apple-silicon
- glm-5.3-flash
- glm5_next
- mtp
- dflash2
- speculative-decoding
- abliterated
- uncensored
- derisked
- 4-bit
pipeline_tag: image-text-to-text
library_name: mlx
GLM-5.3-Flash-Abliterated-MLX-4bit
A high-quality abliterated 4-bit build of GLM-5.3-Flash, optimized for Apple
Silicon with oMLX.
Lineage: zai-org/GLM-5.3-Flash
→ Blackfrost-Research/GLM-5.3-Flash-DERISKED-BF16
(abliterated) → this build (4-bit oQ4e).
The vision tower and the MTP (multi-token-prediction) draft head are both
preserved and working, so the build retains image input and speculative
decoding. Quantized with oMLX to oQ4e: iMatrix-calibrated mixed precision, with
per-tensor bit widths rather than a uniform 4 bits. This model works standalone
on stock oMLX with no extra downloads; the speculative-decoding speedups (MTP
and DFlash2) are optional set-ups described below.
| This build: 4-bit | |
|---|---|
| DFlash2 decode | Short, Thinking-Max: 29.5 - 34.8 tok/s Short, Thinking-High: 39.3 - 51.2 tok/s Long Agentic, Thinking-Max: 33.4 - 33.6 tok/s Long Agentic, Thinking-High: 31.9 - 32.8 tok/s |
| MTP decode | Short, Thinking-Max: 30.4 - 30.9 tok/s Short, Thinking-High: 35.3 - 39.7 tok/s Long Agentic, Thinking-Max: 27.6 - 29.4 tok/s Long Agentic, Thinking-High: 26.7 - 29.6 tok/s |
| AR (no speculative decode) | Short: 29.3 - 29.7 tok/s, flat Long Agentic: 26.5 - 26.6 tok/s, flat |
| Abliteration | 1.3% refusals, measured by Blackfrost-Research on upstream release family |
High is the recommended thinking level for most use cases. As described in the X post from zainhas, testing on Z.ai Code Bench v1.0 shows that GLM-5.3-Flash, High and Max sit at basically the same accuracy, while Max uses about 2× the output tokens (70k vs 140k).
DFlash2 is the fastest option at every context length once its prefix cache works. On the stock GLM adapter the cache is disabled, so DFlash2 re-prefills the whole prompt on every long-context request and MTP or AR win there instead. Our prefix-cache patch (publishing to GitHub shortly; PR pending upstream) fixes that: multi-turn long-context TTFT drops ~46 s to 0.26 s, wall-clock 61.8 s to 15.1 s, and patched DFlash2 then beats both MTP and AR. The warm hit is measured at both temperature 0 and temperature 1. It is not universal - one longer-generation run re-prefilled instead of hitting, and the trigger is still under investigation - so treat the warm numbers as best-case until that is pinned down. Decode rates are unchanged by the patch. oMLX will likely add this natively soon - the fork is only needed if you want DFlash2 with warm caching today.
Wallclock to generate 1024 tokens, short context (temp 0 - temp 1 ranges):
| decode type | Thinking-High | Thinking-Max | High saves |
|---|---|---|---|
| DFlash2 + prefix-cache patch (temp 0) | 21.0 s | 35.4 s | -41% |
| DFlash2 (stock adapter) | 20.8 - 27.5 s | 30.4 - 35.7 s | -10% to -42% |
| MTP | 26.8 - 30.2 s | 34.2 - 34.8 s | -13% to -22% |
| AR | 35.8 - 36.0 s | 35.5 - 35.8 s | ~0% |
Short prompts have no long prefill to cache, so the patch neither helps nor
hurts here - DFlash2 already wins.
Wallclock to generate 512 tokens, Long Agentic context (temp 0 - temp 1 ranges):
| decode type | prompt cache state | Thinking-High | Thinking-Max | High saves |
|---|---|---|---|---|
| DFlash2 + prefix-cache patch (temp 0 measured) | warm | 15.1 s | 15.9 s | -5% |
| DFlash2 (stock adapter) | none (full prefill every request) | 61.8 - 62.1 s | 61.3 - 61.4 s | ~+1% (Max faster) |
| MTP | warm | 22.2 - 24.0 s | 22.3 - 23.4 s | mixed, ±5-8% |
| AR | warm | 24.1 - 24.8 s | 24.1 - 25.1 s | ~0% |
The patched row is measured at temperature 0; the warm hit itself is
confirmed at both temperatures (0.26 s TTFT each). One longer-generation run
missed and re-prefilled, so the warm figures are best-case pending that
investigation.
At equal output length; in real use High's advantage roughly doubles again, since Max generates about 2× the tokens per answer.
| download size | 173 GiB across 35 shards |
| resident when serving | ~190 GiB |
| tensors | 3057 (59 MTP · 347 vision) |
| base model | 320B parameters, ~18B active per token (MoE, 8 of 288 experts + 1 shared) |
| architecture | glm5_next: 45 layers, hybrid KDA (recurrent) + DSA (sparse attention), native multimodal |
| context | 1,048,576 tokens (set your serving window to what your RAM allows) |
⚠️ Intended use and terms
Published for experimental AI research and AI-safety research only:
red-teaming, refusal and alignment research, interpretability, and quantization
or inference research.
It must not be used for anything illegal, in any jurisdiction, under any
circumstances. The removal of refusal behavior is a research property of the
artifact, not an invitation, a permission, or an endorsement.
You are solely responsible for your use of it and for complying with
applicable laws and upstream licence terms. It ships with essentially no safety
refusals: assume it will comply with any instruction, including harmful ones.
Run it locally on hardware you control. Do not expose it as a public endpoint
or deploy it to untrusted users.
Performance
Measured on an M3 Ultra Mac Studio (256 GB) with oMLX 0.6.3. Two workloads:
short (a code-continuation prompt, 1024 tokens generated) and agentic
(a real 14-message tool-calling transcript, ~20.4k GLM tokens of context, 512
generated) - the same benchmark used for the sibling Qwen builds. TTFT counts
the first token of reasoning, not just the answer. Temp 1.0 rows use the
model's recommended sampling (top_p 0.95).
GLM-5.3-Flash reasons at three levels - low, high, and max (the
default) - set via reasoning_effort. There is no native thinking-off: the chat
template always opens a <think> block. The "off" rows below force the block
closed (<think></think> prefix via the completions API), which is the only
true no-thinking mode this model has.
Decode throughput below (DFlash2 rows use the default adaptive verify; see the
tuning note for strict-verify, which is faster on short context). TTFT is
0.85–1.0 s on short prompts. At 20.4k context, cold prefill runs ~445 tok/s
(~46 s); AR and MTP then ride the shared prompt cache (~4.8 s warm), while
DFlash2 on the stock adapter repays the full prefill on every request (see
limitations).
SHORT context (code prompt, 1024 tokens generated)
AR baseline (no speculation): 29.3–29.7 tok/s, flat across all thinking levels and temperatures.
| Thinking level | decode type | decode tok/s (temp 0) | gain over AR (temp 0) | decode tok/s (temp 1) | gain over AR (temp 1) |
|---|---|---|---|---|---|
| Max | DFlash2 | 29.5 | −0.7% | 34.8 | +18.4% |
| Max | MTP | 30.9 | +4.0% | 30.4 | +3.4% |
| High | DFlash2 | 51.2 | +74.1% | 39.3 | +34.1% |
| High | MTP | 39.7 | +35.0% | 35.3 | +20.5% |
| Low | DFlash2 | 49.3 | +66.6% | 48.2 | +63.4% |
| Low | MTP | 40.2 | +35.8% | 36.5 | +23.7% |
| Off* | DFlash2 | 54.0 | +83.7% | 50.2 | +71.3% |
| Off* | MTP | 39.8 | +35.4% | 37.3 | +27.3% |
Long Agentic context (real 14-message tool transcript, ~20.4k tokens, 512 generated)
AR baseline (no speculation): 26.5–26.6 tok/s, flat across all thinking levels and temperatures.
| Thinking level | decode type | decode tok/s (temp 0) | gain over AR (temp 0) | decode tok/s (temp 1) | gain over AR (temp 1) |
|---|---|---|---|---|---|
| Max | DFlash2 | 33.6 | +26.8% | 33.4 | +26.0% |
| Max | MTP | 29.4 | +10.9% | 27.6 | +4.2% |
| High | DFlash2 | 31.9 | +20.4% | 32.8 | +23.8% |
| High | MTP | 26.7 | +0.8% | 29.6 | +11.7% |
| Low | DFlash2 | 33.0 | +24.1% | 29.3 | +10.6% |
| Low | MTP | 28.7 | +7.9% | 27.5 | +3.8% |
| Off* | DFlash2 | 31.5 | +18.4% | 32.7 | +22.9% |
| Off* | MTP | 29.4 | +10.5% | 26.7 | +0.4% |
* No native thinking-off exists in the GLM-5.3 template - Off rows use the
forced-close method (<think></think> prefix via the completions API).
Recommended configurations (measured):
| use case | config | decode |
|---|---|---|
| Short / interactive | DFlash2, verify_mode: "dflash" (strict) |
56.4 off · 51.8 low · 50.6 high · 33.2 max |
| One-shot long context | DFlash2, verify_mode: "adaptive" |
33.6 at 20.4k |
| Repeated long-context chat | DFlash2 + prefix-cache patch (0.32 s warm TTFT), or MTP on stock patches (4.8 s warm TTFT) |
Strict verify wins short prompts (full speculation blocks pay off on
high-acceptance content, +4–13 % over adaptive); adaptive wins past ~8k
context. Both verify modes produce byte-identical output to each other.
Quality: plain decoding is bit-deterministic (verified across restarts).
Speculative output is deterministic and greedy-equivalent up to
floating-point tie-breaks - roughly one benign fork per few hundred
characters where a near-tie resolves differently and both continuations are
valid; it is not bit-identical to non-speculative output. Verified by reading
divergences, not just hashing.
Speculative decoding: two optional speed-ups
The model works standalone. For faster decode you can enable MTP (the
draft head inside this repo - no extra download) or DFlash2 (a separate
2.3 GB drafter - bigger speed-up). Both are greedy-equivalent up to
floating-point tie-breaks (see Quality above) - a different but equally valid
answer, not a worse one.
Both currently require two oMLX patch sets on 0.6.3, from upstream PR
branches (#3228 - GLM MTP
runtime - and #3261 - GLM DFlash2
adapter; #3261 is maintained as a closed branch). Follow each PR's install
notes, then:
MTP - in ~/.omlx/model_settings.json under this model's entry:
{"mtp_enabled": true}
DFlash2 - download the drafter and point the settings at it:
hf download incoai/GLM-5.3-Flash-DFlash2 --local-dir ~/omlx-models/GLM-5.3-Flash-DFlash2
{
"dflash_enabled": true,
"dflash_draft_model": "~/omlx-models/GLM-5.3-Flash-DFlash2",
"dflash_block_size": 8,
"dflash_verify_mode": "dflash",
"dflash_draft_quant_enabled": true,
"dflash_draft_quant_weight_bits": 8,
"dflash_draft_quant_group_size": 64
}
Use "dflash_verify_mode": "adaptive" instead if your prompts routinely
exceed ~8k tokens. The weight_bits: 8 drafter quantization is measured
fastest on this build - it cuts draft cost with zero acceptance loss (2-bit
collapses; leave it at 8). Restart the server after changing settings. Enable
one speculation mode at a time.
A note on expectations: headline speculative-decoding numbers you may have
seen elsewhere (e.g. ~80 tok/s) come from synthetic benchmarks whose prompt is
one paragraph repeated hundreds of times - the easiest possible input for a
drafter. The numbers on this card are from real prompts. A stock-weights
conversion measured with the same recipe and settings splits the lead with
this build by thinking mode: stock decodes notably faster at default
max-effort thinking (up to +29% short-context with DFlash2), while this build
is faster at thinking off/high/low (+2 to +10%) - which includes all the
fastest configurations on this card. Raw non-speculative speed is identical
between the two.
Abliteration and refusal surface
Refusal behaviour was removed at the weight level in the upstream BF16 by
Blackfrost-Research (their "DWM" method; the MTP layer was transplanted back
from the clean checkpoint). No persona or system prompt ships with this
build - the template is the stock GLM-5.3 template plus multimodal/tool-call
fixes, and it injects nothing.
Blackfrost-Research's reported figures for the release family, measured on
their behavior-matched NVFP4 deployment checkpoint with R1-HARMFUL-BENCH-450
under a bare chat configuration (thinking enabled, maximum reasoning effort,
temperature 1.0, top-p 0.95, top-k omitted, max 16,384 output tokens;
responses reviewed after generation to separate actual refusals from
false-positive string matches):
| Evaluation slice | Final judged refusals |
|---|---|
| Harmful prompts | 4 / 300 (1.3%) |
| Full suite | 5 / 450 (1.1%) |
| API errors | 0 / 450 |
These values are upstream measurements of a different deployment checkpoint,
not of this build. Refusals were spot-checked after quantization (the derisked
behaviour survives), but no systematic refusal benchmark has been run on this
4-bit build. "Abliterated" describes the upstream weights, not a property this
conversion re-measured.
Quick start
Requires oMLX on Apple Silicon (256 GB
recommended).
hf download grant-ai/GLM-5.3-Flash-Abliterated-MLX-4bit \
--local-dir ~/omlx-models/GLM-5.3-Flash-Abliterated-MLX-4bit
omlx serve
curl http://127.0.0.1:8153/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "GLM-5.3-Flash-Abliterated-MLX-4bit",
"messages": [{"role": "user", "content": "Explain MoE routing."}],
"max_tokens": 1024, "temperature": 1.0, "top_p": 0.95}'
Settings that matter
| setting | value |
|---|---|
| sampler | temperature 1.0, top_p 0.95 (from the model's generation_config.json) |
| reasoning effort | chat_template_kwargs: {"reasoning_effort": "low" | "high"} - anything else means max (the default) |
| thinking off | not natively supported - the template always opens <think>. True off requires the completions API with a <think></think> prefix (see Performance) |
| multi-turn chat | pass chat_template_kwargs: {"clear_thinking": true} per the upstream model card |
| MTP / DFlash2 | optional; see Speculative decoding above |
reasoning_effort is the biggest lever on time-to-answer: it controls how many
reasoning tokens are generated before the answer, while plain decode speed
stays flat. With speculation enabled, per-token speed depends on how
predictable the generated text is, not on the effort setting itself - on
code, off/low/high all decode far faster than max (whose deliberative
reasoning drafts poorly), while on methodical math reasoning max can be the
fastest. Note that low is a soft request the model sometimes ignores, not a
hard budget.
Technical details
| architecture | glm5_next - 45 layers: 34 KDA (gated-delta recurrent) + 11 DSA (sparse MLA), 288-expert MoE, mHC hyper-connections, vision tower |
| quantization | oQ4e mixed precision, affine, group size 64 base, per-module bit-width overrides |
| iMatrix | 667 modules calibrated from the model's own activations; the 59 MTP-head tensors are quantized without iMatrix |
| MTP head | preserved (stored upstream as layer 45, remapped to language_model.mtp.*) |
| vision | preserved, 347 tensors, verified with a real image request |
| runtime | oMLX 0.6.3 |
How it was made
- Abliterated BF16 weights from Blackfrost-Research (599 GB, 120 shards),
derived from zai-org/GLM-5.3-Flash. - iMatrix measured from the model's own activations on a disk-built
calibration proxy (the source exceeds the in-memory calibration limit). - Body quantized with iMatrix-calibrated oQ4e; the MTP head preserved through
conversion with a glm5_next-specific sanitize patch (stock oMLX 0.6.3 drops
it) and quantized alongside the body. - Vision tower carried through and verified on a real image, not a loader
check. Coherence, refusal spot-checks, and speed all verified by reading
actual outputs.
Limitations
- Capability benchmarks have not been run; conversion correctness and
throughput were verified. - Refusal behaviour was not systematically re-measured on this build.
- With DFlash2 on the stock adapter, every request pays the full prompt
prefill (~46 s at 20k tokens): the GLM adapter disables the prefix cache
because GLM's hybrid cache state is not serializable there. Our
prefix-cache patch adds that serialization (multi-turn TTFT 0.32 s
measured at both temperatures; publishing to GitHub shortly, PR pending
upstream). The hit is not yet guaranteed on every turn - one
longer-generation run re-prefilled instead - so MTP remains the safer choice
for repeated long-context chat until that is resolved. - Speculative output is greedy-equivalent but not bit-identical to plain
decoding (rare floating-point tie-break forks; both continuations valid). - No native thinking-off exists in the GLM-5.3 template.
Credits
- Z.ai (zai-org) for GLM-5.3-Flash.
- Blackfrost-Research for the
abliterated BF16 weights this build was quantized from. - inco.ai for the GLM-5.3-Flash-DFlash2
drafter. - oMLX and the authors of PRs #3228 and #3261 for the GLM runtime and adapters.
Disclaimer and licence
This is an independent, unaffiliated conversion. It is not endorsed by,
developed with, sponsored by, or associated with Z.ai, Blackfrost-Research,
inco.ai, or oMLX. None of them reviewed, approved, or contributed to this
repository, and none of them bear any responsibility for it.
It does not represent GLM-5.3-Flash. The behavior of this model is not the
behavior of the official release: refusal behavior has been removed by a
third-party weight modification. Do not treat any output as representative of,
or attributable to, the upstream model or any party named here. "GLM" is used
solely to identify the base architecture.
Changes made to the upstream work: quantized to mixed-precision oQ4e with
iMatrix calibration; MTP draft head preserved and quantized; vision tower
preserved; no template or persona changes - the source checkpoint's template
ships as-is; no weights modified beyond quantization (the abliteration itself
is upstream work).
Licence: MIT, inherited from the upstream model.