GLM-5.2 MLX Conversion (Uncensored)

MLX-format conversions of zandenAI/GLM-5.2-FP8-Uncensored - an FP8 block-scaled
MoE-DSA (Mixture-of-Experts with DeepSeek-V3-style sparse-attention indexer) model,
141 safetensors shards, architecture GlmMoeDsaForCausalLM (MLA attention +
256 routed experts x 8 active + 1 shared expert + DSA indexer + 1 MTP draft layer).

This repo is the Q6+down8 mixed variant. Sibling repos:

• jeweled/GLM-5.2-Uncensored-MLX-Q8 - uniform 8-bit (master, ~736 GB)
• jeweled/GLM-5.2-Uncensored-MLX-Q6down8 - mixed: base 6-bit, MoE down_proj 8-bit (~736 GB)
• jeweled/GLM-5.2-Uncensored-MLX-Q4 - uniform 4-bit (~418 GB)

Source & credit

• Source model: zandenAI/GLM-5.2-FP8-Uncensored (FP8 weights; convert on your own hardware).
• Conversion was only possible thanks to the glm_moe_dsa indexer-sharing fix from
  pcuenca's glm-moe-dsa-indexer-sharing branch
  (mlx_lm/models/glm_moe_dsa.py). Without it, mlx_lm convert throws
  ValueError: Missing 285 parameters because the DSA sparse-indexer tensors don't map under
  released mlx-lm 0.31.3. That fix tracks ml-explore/mlx-lm PR #1410
  (not yet merged). Discovered via mlx-community/GLM-5.2-DQ4plus-q8 discussion #1.
  Drop that patched glm_moe_dsa.py into your mlx_lm/models/ dir before converting/loading.
• Toolchain: mlx-lm==0.31.3 + mlx==0.32.0 + Python 3.11, Apple Silicon (512 GB Macs, served via exo).

Variant specifics (this repo: Q6+down8 mixed)

• Mixed precision: base weights 6-bit, MoE down_proj (the .w2.weight tensors, the quality-sensitive expert down-projection) pinned to 8-bit, group_size 64. Achieved ~8.5 bits-per-weight average. Custom quant_predicate in quantize_model.

Reproduction

# 1. patch the model class (see "Conversion was only possible thanks to" above)
curl -L https://raw.githubusercontent.com/pcuenca/mlx-lm/glm-moe-dsa-indexer-sharing/mlx_lm/models/glm_moe_dsa.py \
  -o $(python -c "import mlx_lm,os;print(os.path.dirname(mlx_lm.__file__)+'/models/glm_moe_dsa.py')")

# 2. convert (from the local FP8 source snapshot dir)
mlx_lm convert --hf-path <GLM-5.2-FP8-Uncensored snapshot> --mlx-path ./out \
  -q --q-bits 8 --q-group-size 64 --q-mode affine

# for the Q6+down8 mixed variant, load the Q8 output lazily and re-quantize
# with a predicate pinning MoE down_proj (w2) to 8-bit, everything else to 6-bit:
#   from mlx_lm.utils import load, save, quantize_model
#   m,tok = load("./q8_out", lazy=True)
#   def pred(path, mod):
#       return {"bits":8,"group_size":64} if path.endswith(".w2.weight") else {"bits":6,"group_size":64}
#   qm,qc = quantize_model(m, {"quantization":{"group_size":64,"bits":8,"mode":"affine"}}, group_size=64, bits=None, quant_predicate=pred)
#   save("./q6down8", "./q8_out", qm, tok, json.load(open("./q8_out/config.json")))

Files

• model-00001-of-00NNN.safetensors ... (MLX quantized weights)
• config.json (MLX quant config under quantization)
• tokenizer* / chat_template.jinja (passed through from source)