base_model:

• zai-org/GLM-5.3-Flash
• orcarouter/GLM-5.3-Flash-Uncensored-FP8
  base_model_relation: quantized
  library_name: exllamav3
  pipeline_tag: image-text-to-text
  tags:
• glm
• glm-5.3
• glm5_next
• moe
• fp8
• exl3
• exllamav3
• quantized
• 2.0bpw
• k2
• mul1
• uncensored
• multimodal
• mtp
• speculative-decoding
  language:
• en
• zh
• es
• fr
• ru
• ja
• ko
• de
  license: mit
  model-index: []
  widget: []

GLM-5.3-Flash · Uncensored · EXL3 2.00 bpw (mul1, K=2) — for SM90 / SM100 / SM120

306 GiB block-FP8 MoE → 87 GiB. Routed experts at exactly 2.00 bpw; attention, shared
experts and dense MLP are quantised lightly to 4–6 bpw. Runs on stock exllamav3 —
no patched kernels, no forks, no PRs.

Sibling repo bullix/GLM-5.3-Uncensored-EXL3-2.0bpw-v100
holds the mcg pack for SM70 / V100. Same source revision; the difference is the target engine.

Why the sidebar count is wrong. Hugging Face counts tensors in the safetensors headers, and in
an EXL3 pack the trellis index tensor is int16 of shape [in/16, out/16, 32] — so the counter
reads elements of I16 and reports them as parameters. Those are codebook indices, not weights.
The model itself is unchanged at 321.3 B parameters; what this repo stores is a 2-bit encoding
of them. Every routed expert weight is still there, packed 8 bits per 4 parameters.

Uncensoring inherited whole from orcarouter. This repo changes numerics only.
License MIT, inherited from GLM-5.3-Flash.

Why the bits are where they are

The mul1 / K=2 shape is what stock exllamav3 reads natively, so nothing has to be left at FP16
for compatibility. That frees the budget to be spent where it matters:

Measured Final bitrate (excluding head): 2.09 — the +0.09 is exactly the attention / shared /
dense / lm_head load. Average across the directory ≈2.09 bits/param.

The command

python convert.py \
  -i /data/glm_src -o /data/glm_exl3_k2 -w /data/glm_work \
  -b 2.0 -hq 2 -hb 6 -mb 8 -vb 16 -eb 16 -cb mul1 -mae 0.01

refit_scales runs automatically after LDLQ because a Hessian is present — it is not a flag.

What the converter decides

Printed before touching a single weight, verbatim from the run log:

 -- Quantization strategy, summary:
     - 2 bpw:
        model.language_model.layers.*.mlp.experts.*.down_proj          12,096 tensors
        model.language_model.layers.*.mlp.experts.*.gate_proj          12,096 tensors
        model.language_model.layers.*.mlp.experts.*.up_proj            12,096 tensors
     - 4 bpw:
        model.language_model.layers.*.mlp.{down,gate,up}_proj          3 each
     - 6 bpw:
        lm_head                                                             1 tensor
        model.language_model.layers.*.mlp.shared_experts.{down,gate,up}_proj  42 each
        model.language_model.layers.*.self_attn.indexer.wq_b               11 tensors
        model.language_model.layers.*.self_attn.kv_a_proj_with_mqa         11 tensors
        model.language_model.layers.*.self_attn.o_proj                     45 tensors
        model.language_model.layers.*.self_attn.q_a_proj                   11 tensors
        model.language_model.layers.*.self_attn.q_b_proj                   11 tensors
        model.language_model.layers.*.self_attn.qkv_proj                   34 tensors
     - 8 bpw:
        model.language_model.layers.*.eh_proj                               1 tensor
        model.language_model.layers.*.mlp.experts.*.{down,gate,up}_proj   288 each  ← MTP layer
        model.language_model.layers.*.mlp.shared_experts.{down,gate,up}_proj  1 each ← MTP layer
        model.language_model.layers.*.self_attn.{indexer.wq_b,kv_a_proj_with_mqa,
                                                o_proj,q_a_proj,q_b_proj}     1 each ← MTP layer

Calibration

250 rows × 2048 tokens = 512 000 tokens, taken from the corpus that ships inside exllamav3
(exllamav3/conversion/standard_cal_data/): wiki 50 / c4 20 / code 20 / random 20 / multilingual 10 /
technical 10 / tiny 5. Nothing was downloaded.

-mae 0.01 builds a separate Hessian for each of the 288 experts from a mixture of "all
calibration tokens" and "the tokens the router actually sent to this expert", ε = 0.01. Cap is
262 144 of 512 000 tokens per layer; with 288 experts this was never hit, no layer truncated.
The corpus is read once per layer, with state propagating through the already-quantised layers
above, so layer L is calibrated on inputs resembling what it will actually receive.

Expert matrix layout

<name>.trellis  int16  [in/16, out/16, 32]   K = 32/16 = 2
<name>.suh      fp16   [in]
<name>.svh      fp16   [out]
<name>.mul1     int32  scalar               0x83DCD12D

There is no .mul1/.mcg mix: mul1 holds one single value across all 37 420 quantised
matrices, and .mcg does not appear anywhere in this repo.

Measured accuracy

Produced 2026-11-02 against this exact pack, on one H100 NVL 94 GB (SM90) with
exllamav3 v1.6.6 at bd1f519. Raw numbers ship as metrics_glm_k2_mul1.json.

KL and perplexity are measured with eval/model_diff.py, which loads both models one layer
at a time, so the 306 GiB parent fits alongside the pack on one card. Panel: wikitext-2,
100 rows × 2048 = 204 800 positions. The same tool, panel and parent were used for the
mcg pack, so the two are directly comparable.

Structural acceptance

verify_out.py --mtp-k 8 → ИТОГ: OK (FAIL=0, WARN=0)

Perplexity

Retention by perplexity: 77.2 %. Both row counts are reported rather than only the
favourable one.

KL divergence (100 rows × 2048 = 204 800 positions)

KL by confidence bucket (B top-prob):

Where the error comes from

Per-layer relative error of each layer's output against the parent (rfn_err from model_diff),
split by attention type. Layers 0–44:

Restricted to the 42 MoE layers (3–44, no dense layers 0–2):

Layer error stays small through layer 16 (rfn 0.025 → 0.336) and then jumps at layer 17
(0.679), staying high for the rest of the stack. Mean for layers 0–16 is 0.189, for 17–44 it
is 0.807.

That jump is not a sparse-MLA effect. Layers 16 and 17 are both KDA. The same jump appears
in the mcg pack at the same place. It is the accumulation of 2-bit expert error along the
residual stream, not the indexer.

Comparison with the mcg pack

Both packs come from the same parent revision, are measured with the same tool
(eval/model_diff.py), on the same panel (wikitext-2, 100 × 2048 = 204 800 positions).
Sizes are the sum of file sizes reported by the HF API for each repo.

How to read this table. mul1 is smaller and more accurate — 5.9 GiB less on disk for
0.085 lower KL. But the gain is a uniform offset: KL, perplexity and per-layer error all
improve by roughly the same 15 %, and the improvement is identical across attention types
(sparse −15.1 %, KDA −14.2 %).

That refutes an earlier hypothesis. The mcg pack's higher error was attributed to its eleven
sparse-MLA layers and to the FP16 requirement. If that were the mechanism, mul1 would have
recovered mostly on those eleven layers and barely at all on KDA. It did not. The recipe change
is a constant improvement on every layer, i.e. it comes from the codebook and the bit allocation,
not from a specific broken subsystem.

The 6.4 % size saving is explained mechanically: mcg must leave attention, shared experts,
dense MLP and lm_head at FP16 (V100 kernel contract), while mul1 stores them at 4–6 bpw.
Nothing was dropped — the experts are identically 2 bpw in both.

Reading the numbers honestly

KL 0.478 is still high for 2 bpw. The nearest published comparison point on this
architecture is philbert440/GLM-5.3-Flash-EXL3 at KL 0.132 and 87.8 % top-1 agreement
against its own parent — different data, so not a clean comparison, but the same architecture
family. This pack is roughly 3.6× worse on that axis. That is not a normal result for
2 bpw and is not presented as one.

What is known: the error is spread evenly across depth after layer 16, is not specific to
sparse-MLA, and comes from the 2-bit expert weights — the non-expert layers feeding them are
quantised only lightly. What is not known: how much of the remaining gap a higher K would
close.

Cost of raising K — exact calculation from headers

If a lower KL is worth the bytes, K has to be raised contiguously from layer 16 downward
(not selectively on the eleven sparse layers — see above). This is computed over all 37 420
safetensors headers, not estimated.

Why the numbers are exact. A trellis tensor is int16 of shape [in/16, out/16, 16K],
so its size is exactly in·out·K/8 bytes and the trellis rate is exactly K bits per weight.
Checked against all 37 420 header entries: 0 deviations. On top of that each matrix carries
suh[in] + svh[out] in fp16 and a 4-byte marker, so the full rate is

bpw_full(K) = K + (16·(in+out) + 32) / (in·out)

For a 4096×2048 matrix that is 2.011723 at K=2 and 3.011723 at K=3 — both match the
measured pack size to six decimals.

Per layer. Each of the 42 MoE layers holds exactly 7 247 757 312 weights (identical in
all 42). Going from K=2 to K=3 adds one bit per weight, i.e. 7 247 757 312 / 8 bytes:

+0.84375 GiB per layer, exactly

Control: recomputing the pack from headers at K=2 gives 76 547 503 872 bytes of expert data —
identical to the actual file contents, byte for byte.

Layers 0–2 are dense and hold no experts, so the lowest possible threshold is layer 3.

"Whole-model bpw" is pack size × 8 ÷ 321 323 031 390 real parameters (computed from the
headers, matching the 321.3 B stated above). It sits above the expert rate because it also
counts the FP16 embed_tokens, vision tower, MTP head and the unquantised norms.

What this is and is not. These size numbers are arithmetic and are exact. The KL these
variants would produce is unknown — nothing here was run. The +0.84 GiB per layer is about
6× the "+4.2 GiB selective" figure quoted earlier in this project, which assumed a bump on
only the eleven sparse layers; that assumption is now known to be wrong.

Practical reading: the pack loads on stock exllamav3, produces coherent output, and keeps the
right token first in 79 % of positions. On tokens where the model is confident (top-prob > 0.95,
29.6 % of positions) KL falls to 0.055 — where it knows the answer, it keeps it. It is
not a drop-in replacement for the FP8 parent on quality-sensitive work.

Running it

pip install exllamav3==1.6.6     # or clone at commit bd1f519

from exllamav3 import Model, Config, Tokenizer, Generator

model = Model.from_config(Config.from_directory("/path/to/this/repo"))
model.load()
tok   = Tokenizer.from_config(Config.from_directory("/path/to/this/repo"))
gen   = Generator(model, tok)

Vision, MTP and the tokenizer sidecars ship inside the pack; nothing else needs downloading.
No kernel patches, forks or PRs are required — this is the format stock exllamav3 reads.

What this repo is not

• Not a fresh model. Behaviour, refusals, style — all inherited from
  orcarouter/GLM-5.3-Flash-Uncensored-FP8, unmodified.
• Not the V100 pack. Same source, different recipe (mcg, FP16 non-experts, 128×8192
  calibration) — see the sibling repo.
• Not a low-KL pack. 0.478 is measured, not assumed, and it is high; see Reading the numbers
  honestly.

Reporting a problem

Include engine name and commit, nvidia-smi --query-gpu=name,memory.total --format=csv, the full
traceback, and config.json's quantization_config. SHA256SUMS covers shards, index and config.