license: mit
base_model:

• orcarouter/GLM-5.3-Flash-Uncensored-FP8
  base_model_relation: quantized
  pipeline_tag: image-text-to-text
  language: [en, zh]
  tags:
• glm
• glm5
• glm-5.3-flash
• moe
• compressed-tensors
• int4
• w4a16
• sglang
• quantized
• abliterated
• uncensored
• not-for-all-audiences

GLM-5.3-Flash-Uncensored-W4A16

INT4 weight-only (W4A16, group-128, symmetric) quantization of
orcarouter/GLM-5.3-Flash-Uncensored-FP8 —
the abliterated (refusal-removed) build of Z.ai's GLM-5.3-Flash.

Produced for, and actually served on, self-hosted SGLang / 4× RTX PRO 6000 Blackwell (SM120).
Every number on this card was measured on that box, and every number is labelled with which weights it
was measured on: unlabelled figures are these weights; anything carried over from this checkpoint's
censored sibling (voska/GLM-5.3-Flash-W4A16 — same
architecture, same quantizer, same config) is marked †. Where something was not measured at all, the
card says so.

† means: measured on the censored sibling, not re-measured on these weights. Those figures are here
because the two builds are identical in architecture, quantizer and serving config — but they are not
measurements of this checkpoint, and this card will not pretend otherwise.

Provenance

zai-org/GLM-5.3-Flash                          (base, MIT, © Z.AI Co., Ltd)
  └─ orcarouter/GLM-5.3-Flash-Uncensored-FP8   (abliteration baked into the official block-FP8 checkpoint)
       └─ voska/GLM-5.3-Flash-Uncensored-W4A16 (this repo — data-free RTN INT4)

The source was pinned at commit
55c78761fb78d0f765e2bd01687900d2fd91b5d0
and confirmed by blob-hash match against the local download tree, not inferred from a directory name.
Licence MIT, carried forward from the base model.

Note: the upstream FP8 repo is gated. If you want to reproduce this quantization from source you will
need to accept its terms first.

What "uncensored" means here, and what was actually tested

This is a derivative of someone else's abliteration. We did not perform the abliteration and we do not
republish its author's claims. The one question we did have to answer — because it was genuinely open —
was whether abliteration survives 4-bit weight quantization at all. It is not obvious that it does: the
quantizer perturbs exactly the weight directions abliteration edits.

Measured on this checkpoint, served:

So: abliteration survives INT4, and this build is less prone to false-positive refusal than its censored
sibling. That is the whole of what was tested. No safety, capability, or agentic benchmark was run — see
Limitations.

⚠️ Read this before you download 178 GiB

As of 2026-08-30 no released or nightly SGLang can load this checkpoint. GLM-5.3-Flash support lives in
sgl-project/sglang#36507, which is still open and
conflicting with main. The DFlash hidden-state-capture fix
(#36708) merged into that PR branch
(xinyuan/glm-5.3-flash-support), not into main. You need a build from #36507 plus the three SM120 patches
below. This is not a config you can copy onto a stock image today.

We would rather open with that sentence than have you find out at the end of the download.

Serving

Hardware this was measured on

4× RTX PRO 6000 Blackwell Max-Q, 96 GB each (SM120, no NVLink, PCIe Gen5 P2P), power-limited to 260 W;
Threadripper PRO 9975WX, 128 GB DDR5; driver 580.159.04-open. Kernel cmdline iommu=off amd_iommu=off
plus options nvidia_uvm uvm_disable_hmm=1 are both required for NCCL P2P on this board.

Numbers below will not transfer to datacenter Blackwell (SM100). SM120 has no tcgen05/TMEM and caps dynamic
shared memory near 99 KB, which is what forces patch 3.

Versions

sglang 0.0.0.dev1+gd6ab04bdf1 (built from #36507 + patches) · torch 2.13.0+cu130 · CUDA 13.0 ·
transformers 5.16.0.dev0 · compressed-tensors 0.18.0 · flashinfer 0.6.17 · driver 580.159.04-open

A launch line with no version pin rots inside a month — SGLang flag names churn.

Three SM120 patches, all still required

1. SGLANG_OPT_DEEPGEMM_HC_PRENORM=0 (env). Hyper-connection prenorm calls DeepGEMM unconditionally and
   SM120 has no TMEM/tcgen05 → NameError: deep_gemm.
2. --dsa-prefill-backend tilelang --dsa-decode-backend tilelang. The default decode path routes to
   trtllm-gen FMHA, which reports "Unsupported architecture" on SM120.
3. num_stages=1 in the TileLang sparse-MLA schedule — a source patch, not a flag. Upstream uses
   num_stages=2, which asks ~148 KB of dynamic shared memory; SM120 caps it near 99 KB. Not upstream yet.

KV dtype: keep bf16

--kv-cache-dtype fp8_e4m3 produces garbled output, or a single token and then a stop. The cause is a
FlashInfer SM120 bug — the backend is missing FP8 dequantization scales, so the ragged+paged split path reads
cached KV back without undoing the scale division. Independently reported on non-GLM models on the same
hardware (sglang#19603). Expect it on any SM120 model
on this backend, not just this one.

The launch line, exactly as run

docker run --rm --name glm53 --gpus all --network host --ipc=host \
  -v /path/to/GLM-5.3-Flash-Uncensored-W4A16:/model:ro \
  -v /path/to/GLM-5.3-Flash-DFlash2:/draft:ro \
  -v /srv/triton-cache:/triton-cache \
  -e TRITON_CACHE_DIR=/triton-cache \
  -e SGLANG_OPT_DEEPGEMM_HC_PRENORM=0 \
  -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
  YOUR_SGLANG_IMAGE_BUILT_FROM_36507_PLUS_THE_3_PATCHES \
  python3 -m sglang.launch_server \
    --model-path /model --served-model-name glm-5.3-flash \
    --tp 4 --context-length 1048576 \
    --dsa-prefill-backend tilelang --dsa-decode-backend tilelang \
    --kv-cache-dtype bfloat16 \
    --speculative-algorithm DFLASH \
      --speculative-draft-model-path /draft \
      --speculative-num-draft-tokens 8 \
    --mem-fraction-static 0.90 \
    --chunked-prefill-size 4096 \
    --max-running-requests 12 \
    --schedule-policy lpm \
    --mamba-radix-cache-strategy extra_buffer \
    --mamba-track-interval 2048 \
    --cuda-graph-max-bs 16 \
    --reasoning-parser glm45 \
    --tool-call-parser glm47 \
    --enable-hierarchical-cache \
    --hicache-ratio 0.5 \
    --hicache-write-policy write_through \
    --enable-metrics --trust-remote-code \
    --host 0.0.0.0 --port 8100 \
    --disable-shared-experts-fusion

Drop the three --speculative-* flags and the /draft mount for the autoregressive configuration.

--disable-shared-experts-fusion is mandatory, not stylistic. Without it this architecture emits fluent
gibberish and never stops, on any quantized build. determine_num_fused_shared_experts() sizes the MoE at
n_routed_experts + 1 = 289 slots, then a separate wrapper gate — evaluated ~2 minutes later during weight
loading — decides the config does not support fusion and never populates slot 288. Every token then routes
through an uninitialized expert. Harmless on the FP8 release; fatal once expert slots are INT4-packed.

Parsers are not optional. Without --reasoning-parser glm45 the chain-of-thought lands in content with
a stray </think>; without --tool-call-parser glm47 every agentic client gets a 400 while plain chat looks
perfectly healthy. Both failure modes shipped to production here before they were caught. /health says
nothing about parsers.

--chunked-prefill-size 4096 is a ceiling, not a tuning knob. 8192 crashed the scheduler on a 305K-token
cold prefill: the DeepGEMM MQA-logits buffer is chunk × max_seqlen_k × ~1.14 bytes, so at 8192 against a
1.8M-token pool it wants 4.07 GB — more than this box ever has free. Do not lower it either; see the
chunk-size table on the censored sibling's card.

The unit, as deployed

systemd, secrets stripped. MODEL/DRAFT/IMAGE are passed as Environment= to a wrapper script that
execs the docker run above.

[Unit]
Description=GLM-5.3-Flash-Uncensored (W4A16, TP=4) on SGLang with SM120 fixes
After=docker.service network-online.target
Requires=docker.service
# This box has produced a 206,804-restart loop. Rate-limit restarts.
StartLimitIntervalSec=900
StartLimitBurst=4

[Service]
Type=simple
ExecStartPre=-/usr/bin/docker rm -f glm53
ExecStart=/usr/local/bin/glm53-run.sh
ExecStop=/usr/bin/docker stop -t 90 glm53
Restart=on-failure
RestartSec=45
# 178 GiB (191 GB) across 4 GPUs + CUDA graph capture. Model load takes minutes.
TimeoutStartSec=1500
TimeoutStopSec=150
Environment=MODEL=/srv/models-cache/GLM-5.3-Flash-Uncensored-W4A16
Environment=DRAFT=/srv/models-cache/GLM-5.3-Flash-DFlash2
Environment=EXTRA_ARGS=--disable-shared-experts-fusion

[Install]
WantedBy=multi-user.target

TimeoutStartSec matters: startup to healthy is ~5m20s with the draft. A shorter timeout turns a normal boot
into a restart loop.

Expected health at startup

If your deploy is right, you should see approximately this. If max_total_num_tokens comes up materially
below it, your --mem-fraction-static or context length differs — the pool, not the --context-length
flag, is what enforces the input ceiling.

The with DFlash2 column is this checkpoint, read off /metrics and the boot log on the live
resident (weights 42.03 GB/GPU and the 98.30 s load are from this build's own startup). The
autoregressive column is † — we never ran these weights without a draft, because doing so would have
meant a second restart of a box that serves live traffic for a number we already had. Given the KV pool,
mamba pool and input ceiling reproduced the sibling exactly on the DFlash2 side, the autoregressive
column is expected to transfer — but expected is not measured.

Measured throughput

Single-stream figures are idle-gated on sglang:kv_used_tokens == 0 and token-accurate from
usage.completion_tokens.

Three measurement traps — these numbers will not reproduce if you hit them:

• Never count SSE chunks under speculative decoding. One chunk can carry several accepted tokens. Read
  tok/s from usage.completion_tokens.
• sglang:num_running_reqs is not a live concurrency signal. It refreshes only on the decode-log tick
  (~1 s here) and never at request admission or completion — it read 1.0 while two requests concurrently
  held KV, and 0.0 while one held 15,104 tokens. Gate idleness on sglang:kv_used_tokens, which moves
  immediately and counts active requests only.
• Aggregate throughput is bimodal by warm-up state. The first run after an idle gap is ~30% faster than
  back-to-back runs, because the HiCache write-through host tier is still draining when kv_used_tokens
  already reads zero. State which one you measured.

A fourth, specific to comparing models: under speculative decoding tok/s is comparable across configs on
one model, but not across models — a tersely-answering model posts a different tok/s at an identical
engine step rate. The invariant is chunks/s, not tokens/s.

Speculative decoding: use a standalone draft, not the MTP head

The whole NEXTN row is † — measured 2026-08-29 on the censored sibling, and never re-run on these
weights. We did not re-measure a configuration we had already established is a large loss, on a serving
box. The DFlash2 accept length and tok/s are this checkpoint; only the 104.5 autoregressive
baseline they improve on is carried over.

The generalisable result: an MTP head's acceptance does not survive quantizing the target; a standalone
draft's does. The MTP head was trained against the full-precision model, and 4-bit weight quantization
perturbs exactly the hidden states it consumes. A draft that only consumes token IDs does not care what
format the target is in. No retune of num_steps / topk / draft-token count rescues a 4.2% accept rate.

The draft is a separate download and we do not redistribute it. incoai/GLM-5.3-Flash-DFlash2 is public
and licensed CC-BY-NC-ND-4.0 — a more restrictive licence than these weights, non-commercial and
no-derivatives. Fetch it from its own repo. block_size: 8 is declared in its dflash_config, which is where
--speculative-num-draft-tokens 8 comes from. If you cannot accept CC-BY-NC-ND, use the autoregressive line
above; it is fully supported and the 104.5 tok/s † column is the honest expectation.

Why W4A16 and not FP8

We served the FP8 source of these exact weights on the same box and rolled it back. It works, but 75.28 GB/GPU
of weights instead of 42.03 leaves an entirely different machine:

Measured directly rather than inferred: a 335,616-token prompt succeeds on FP8, and 351,816 returns
HTTP 400 — Input length (351816 tokens) exceeds the maximum allowed length (347002 tokens).

On 4×96 GB, quantization format is a context-length decision, not just a speed one. Keep resident weights
under ~250 GB or the KV pool stops covering your own prompt distribution.

What is and isn't quantized

Quantized: routed-expert gate_proj / up_proj / down_proj for layers 3–44 only
(42 × 288 × 3 = 36,288 packed tensors). Also 1,050 tensors dequantized FP8→BF16 and 1,432 copied
unchanged.

Left at BF16 (in the ignore list) — each for a reason:

Known inefficiency, not fixed here: the 1,050 non-expert tensors that were blockwise-FP8 in the source are
stored as BF16, doubling their bytes on the always-active path. A parameter census puts attention at ~58% of
per-token decode bytes and routed experts at only ~20%, so re-storing those as FP8 is the obvious next gain
(~13% fewer bytes/token). Not done because SGLang's FusedMoE path reads only the top-level
compressed-tensors format and rejects mixed-precision
(sglang#16276 — fixed for Linear, not MoE).

Method

Streaming tensor-level transform directly from the FP8 source (no BF16 checkpoint was materialised —
dequantized BF16 is 642 GB and does not fit in 384 GB VRAM + 128 GB RAM):

1. dequantize blockwise FP8 (128×128 weight_scale_inv) → fp32
2. per-group symmetric min/max scales via compressed_tensors.calculate_qparams, computed against the
   BF16-rounded scale that is actually stored (quantizing against the fp32 value and then storing BF16
   leaves the checkpoint self-inconsistent at inference)
3. symmetric INT4 group-128, packed 8-per-int32 via compressed_tensors' own packer
4. all other FP8 tensors dequantized to BF16; weight_scale_inv removed

Mean sampled relative L2 error vs the FP8 source: 11.093% (n=200).

This is data-free quantization — not GPTQ, not AWQ, no calibration set, no activation-aware scaling. A
calibrated build should beat it on quality.

It takes 7.2 minutes on a single GPU. 328 GB in, 191 GB out (= the 178 GiB on-disk figure above;
same bytes, decimal vs binary), 432 seconds, one RTX PRO 6000. This is worth
stating plainly because the widespread belief that quantizing a 320B MoE is an overnight job stops people
trying: that belief is true of calibration-based AWQ/GPTQ, which runs forward passes over calibration data,
and false of data-free RTN, which is a pure per-group weight transform. At 0.76 GB/s against a drive that does
~12, it is per-shard compute-bound and trivially shardable across GPUs if anyone needs it faster.

Limitations — read this

• No agentic, coding, or capability benchmark was run. Nobody has published agentic deltas for any
  GLM-5.3-Flash quant, and this card does not claim one. The numbers above are throughput, capacity,
  long-context retrieval, and capability smoke-tests only.
• No KLD or perplexity was measured on this specific checkpoint. The censored sibling's card cites a
  published KLD panel for the format class; that panel is not a measurement of these weights and is not
  repeated here.
• The upstream abliteration's published benchmarks do not transfer to this build and are not reproduced.
  Quantization perturbs the weights; the only abliteration property we verified post-quantization is the
  refusal behaviour in the table above, on 30 prompts total. That is a smoke test, not an evaluation.
• The autoregressive baseline and the entire NEXTN/MTP comparison are carried over from the censored
  sibling (marked † above), not measured on these weights. They are here because the two builds share
  architecture, quantizer and config, and because re-measuring them meant restarting a live box for a
  known answer. Treat them as strong priors, not as measurements of this checkpoint.
• Data-free; a calibrated GPTQ/AWQ build should beat it on quality.
• Measured only on SM120 (RTX PRO 6000 Blackwell Max-Q, power-capped 260 W), TP=4, no NVLink. Weight-only
  INT4 runs the Marlin-class path, not native FP4 tensor cores. The ~260 W cap costs roughly 4% and is not
  what bounds these numbers — kernel/backend selection is.
• Serving requires an unmerged SGLang PR plus three local patches (above). There is no supported stock build.
• Vision is preserved in the weights and the tower is left at BF16, but vision was smoke-tested, not
  benchmarked.

Safety

This is an abliterated / refusal-removed model. It will comply with requests that the base GLM-5.3-Flash
declines. It is tagged not-for-all-audiences deliberately. You own what you deploy it for, and you own the
guardrails you put in front of it — there are none in these weights.

Licence and attribution

MIT, carried forward from zai-org/GLM-5.3-Flash
(© 2026 Z.AI Co., Ltd) through
orcarouter/GLM-5.3-Flash-Uncensored-FP8.
This repository contains only a requantization of that checkpoint; the abliteration is OrcaRouter's work, not
ours. The DFlash2 draft referenced above is a separate repo under a separate, more restrictive licence and is
not redistributed here.