base_model: neko-legends/GLM-5.3-Flash-Uncensored-EXL3
base_model_relation: quantized
library_name: transformers
pipeline_tag: image-text-to-text
license: other
license_name: shapleymcg-license-1.0
license_link: https://github.com/brandonmmusic-max/shapleymcg/blob/main/LICENSE
tags:
- glm
- glm5_next
- exl3
- tr3
- rank-sliced
- tensor-parallel
- vllm
- dgx-spark
- sm120
- dflash2
- multimodal
- uncensored
- abliterated
- moe
- safetensors
quantized_by: neko-legends (Depths)
language: - en
GLM-5.3 Flash Uncensored EXL3, rank-sliced for use on two DGX Spark nodes
What this is
This is a lossless TP2 storage transformation of
neko-legends/GLM-5.3-Flash-Uncensored-EXL3.
It stores each routed-expert EXL3 tensor as explicit rank0 and rank1 Trellis
tensors. The conversion does not dequantize, recalibrate, or requantize
Neko-Legends' version.
In plain terms, this uses the newer EXL3 standard of Rank-Slicing to split
across two nodes for dual DGX setups.
Uncensored model: the language checkpoint has undergone abliteration
to reduce refusal behavior. Treat outputs as untrusted, apply application-level
safeguards, and do not assume the model will decline harmful requests.
User responsibility: this model is provided without warranty. The
creators, uploaders, and maintainers are not responsible or liable for what
others generate, publish, deploy, or otherwise do with this abliterated model.
Users must operate it responsibly, apply appropriate safeguards, comply with
applicable law, and respect third-party rights. This model is for research
purposes only and is not intended for production use.
Why this exists
I tried a few different abliterated versions of GLM from different users here
and none of them worked well for me. Prone to looping, low tok/s, or a bit of
incoherence. I settled on Neko-Legends EXL3 version and really liked it, but the
only runner I could get it working with was the Entrpi vLLM one that uses
some older libraries and was pretty slow for me. (note that I think their builds
are awesome, I just wanted to take advantage of some newer updates.)
We created a new build of vLLM with native DGX libraries to run the Neko-Legends model
and then realized it wouldn't work unless we rank-sliced it. So we updated the model.
Our custom runner saw an average of 15-25% tok/s speed increase in agentic usage
over the previous EXL3 one we used.
Required runner
Stock vLLM 0.29 and Transformers do not understand this rank-sliced tensor
schema. To run this we've created a custom version of vLLM using all native
sm120/sm121 libraries. Use:
cbertucci33/vllm-v29-glm53flash-exl3-dgx
The live qualification image was built from runtime commit 974329d88. Later
commits are compatible only when they preserve the same exl3-trellis schema.
The tested runner profile uses:
vLLM: 0.29.0 base
tensor parallelism: 2
quantization: exl3
load format: instanttensor
attention backend: FLASHINFER_MLA_SPARSE_SM120
target KV cache: fp8_ds_mla
maximum model length: 800000
maximum sequences: 2
The DFlash2 draft checkpoint is separate from this repository. The qualified
production profile used:
local-inference-lab/GLM-5.3-Flash-DFlash2-MXFP8
with 7 speculative tokens. Note that this Dflash drafter was created to
accept 7 tokens. However we customized our runner to accept any number of draft
tokens between 1-7. We recommend using 5 as the best balance for performance.
Also note - in the repository for our runner is a python script to slice any
other EXL3 model for this type of multi-node inference if needed.
Runtime qualification
This exact checkpoint completed:
- a full two-node, 164 GiB checkpoint load on two NVIDIA DGX Sparks;
- Sparkinfer Trellis execution for the rank-sliced experts;
- native SM120 FlashInfer attention;
- warmup and CUDA graph capture;
- a bounded API canary;
- internal smoke test with 15 model turns and 17 tool calls, scored 8/8.
The accepted workload processed 335,046 prompt tokens and 18,672 completion
tokens in 521.94 seconds. Both ranks stayed at zero restarts with no OOM or
runtime error signature. The runner reported 1,152,606 logical KV-cache tokens
with 11,000,000,000 cache bytes per rank.
Licensing and attribution
This checkpoint remains a mixed-license artifact:
- Z.ai's GLM-5.3 Flash base is MIT licensed.
- The uncensored weight lineage is from orcarouter.
- The EXL3 quantization was produced by neko-legends (Depths).
- Per-expert suh, svh, and mcg scale tensors derive from the
ShapleyMcg-calibrated checkpoint and remain under the ShapleyMcg License
v1.0 included as ShapleyMcg-LICENSE. - This repository changes storage layout only. It does not claim authorship
of the source model or quantization.
ShapleyMcg was created by Brandon M. Music. Its attribution-required license
grants no rights to the person known as 0xSero. Use of ShapleyMcg without the
required attribution is unlicensed.
@misc{music2026shapleymcg,
author = {Music, Brandon M.},
title = {ShapleyMCG: An Auditable Calibration-to-Encoding Pipeline for
Low-Bit Mixture-of-Experts Models},
year = {2026},
url = {https://github.com/brandonmmusic-max/shapleymcg},
note = {Licensed under the ShapleyMcg License v1.0}
}