license: mit
library_name: transformers
pipeline_tag: text-generation
tags:

• glm
• moe
• quantization
• nvfp4
• nf3
• mxfp8
• hybrid-quant
• abliterated
• uncensored
  base_model:
• zai-org/GLM-5.2

SuperGLM-5.2-abliterated — MXFP8 / NVFP4 / NF3 Hybrid

A single-node hybrid-quantized build of the abliterated GLM-5.2 (753B, MoE) that fits on
4×96 GB GPUs at tensor-parallel 4, serving the full 256-expert model at 262K context —
weights preserved, refusals removed, reasoning intact.

• ~341 GB on disk, 92 safetensors shards. Runs where the BF16/NVFP4-only builds need 8 cards.
• Uncensored: the abliteration lives in the BF16 non-expert tier and survives quantization
  (measured 0/12 refusals post-build and post-healing).
• Capability retained: GSM8K 94.2% (flexible) / 93.9% (strict) after the healing pass.

Research artifact. It inherits the behavior and the license of its upstreams — read
Provenance and Intended use below before deploying.

What this is

GLM-5.2 is a Mixture-of-Experts model (78 layers, 256 experts, first_k_dense=3,
glm_moe_dsa). This build applies a per-expert mixed-precision scheme so the whole model
fits four cards instead of eight, without dropping any experts (no structural pruning):

A bit_map (75 MoE layers × 256 experts → 4,800 NVFP4 / 14,400 NF3) drives the per-expert
allocation. This reproduces the hybrid scheme published by madeby561 for the stock GLM-5.2
checkpoint, applied here to the abliterated weights.

Healing pass

After quantization, expert weights are error-corrected with GPTQ-style error feedback
(healed against the source BF16, gate/up sharing a Hessian, down-proj reconstructed from the
SwiGLU intermediate). 36,591 matrices (~85%; low-traffic experts left naive) were healed,
cutting per-expert output error from 14.2% → 1.1%. The healing only touches experts, so the
abliteration (non-expert tier) is untouched.

Provenance & credits

This is a derivative work. Full credit to the upstreams:

• zai-org/GLM-5.2 — the original model and architecture.
• Jiunsong/SuperGLM-5.2-abliterated-NVFP4 — the abliterated (refusal-removed) NVFP4 source
  weights this build quantizes from.
• madeby561 — author of the NVFP4 + NF3 + BF16 hybrid quantization scheme and the vLLM
  serving image; the NF3/NVFP4 quantizers and bit_map here reproduce that method.

The build reproduces the two quantizers (NF3 and NVFP4) and reuses the upstream hybrid bit_map;
the NF3 output format was validated byte-exact against the reference implementation before the
full run.

Requirements

• 4× NVIDIA Blackwell GPUs (SM 120 / RTX PRO 6000-class), ~96 GB each (~384 GB total VRAM).
  The NVFP4 experts and the B12X MoE / sparse-MLA kernels are Blackwell-native — this build
  will not run unchanged on Ampere/Hopper.
• Docker + the NVIDIA container runtime.
• The hybrid-aware vLLM image madeby561/vllm-glm52-nvfp4-nf3-hybrid:v3. Its loader reads the
  bit_map and reconstructs each precision tier at load — a stock transformers/vllm load will
  not reconstruct the mixed-precision experts.

Download

HF_HUB_ENABLE_HF_TRANSFER=1 \
hf download kachowtowmater/SuperGLM-5.2-abliterated-MXFP8-NVFP4-NF3-Hybrid \
  --local-dir ./SuperGLM-5.2-abliterated-hybrid

Serving (vLLM, tensor-parallel 4)

Point MODEL_DIR at the downloaded checkpoint. docker-compose.yml:

services:
  glm52:
    image: madeby561/vllm-glm52-nvfp4-nf3-hybrid:v3
    network_mode: host
    ipc: host
    shm_size: 32gb
    init: true
    ulimits:
      memlock: -1
      stack: 67108864
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    volumes:
      - ${MODEL_DIR:?set MODEL_DIR to the downloaded checkpoint dir}:/model:ro
      - vllm-cache:/cache
    healthcheck:
      test: ["CMD-SHELL", "curl -sf http://localhost:8000/health"]
      start_period: 600s
      interval: 15s
    environment:
      CUDA_VISIBLE_DEVICES: "0,1,2,3"
      CUDA_DEVICE_ORDER: PCI_BUS_ID
      CUTE_DSL_ARCH: sm_120a            # Blackwell
      PYTORCH_CUDA_ALLOC_CONF: expandable_segments:True
      NCCL_IB_DISABLE: "1"
      NCCL_P2P_LEVEL: SYS
      GLOO_SOCKET_IFNAME: lo
      TP_SOCKET_IFNAME: lo
      VLLM_USE_B12X_FP8_GEMM: "1"
      VLLM_USE_B12X_MOE: "1"
      VLLM_USE_B12X_SPARSE_INDEXER: "1"
      VLLM_USE_V2_MODEL_RUNNER: "1"
      B12X_W4A16_TC_DECODE: "1"
      B12X_MOE_FORCE_A16: "1"
      VLLM_DCP_GLOBAL_TOPK: "1"
      VLLM_DCP_SHARD_DRAFT: "1"
      VLLM_ENABLE_PCIE_ALLREDUCE: "1"
      VLLM_PCIE_ALLREDUCE_BACKEND: b12x
      VLLM_PCIE_ONESHOT_MAX_BYTES: "65536"
      B12X_DENSE_SPLITK_TURBO: "1"
      XDG_CACHE_HOME: /cache/jit
      CUDA_CACHE_PATH: /cache/jit
      # hybrid loader
      HYBRID_TIER: both
      HYBRID_KEPT: b12x_nf3
      HYBRID_NF3: b12x_nf3
      HYBRID_B12X_MAX_TOKENS: "2048"
      HYBRID_MXFP8_NATIVE: "1"
    command: >
      vllm serve /model
      --served-model-name GLM-5.2 --host 0.0.0.0 --port 8000
      --trust-remote-code --tensor-parallel-size 4
      --decode-context-parallel-size 4 --dcp-comm-backend ag_rs --dcp-kv-cache-interleave-size 1
      --kv-cache-dtype fp8
      --attention-backend B12X_MLA_SPARSE
      --moe-backend b12x
      --load-format safetensors
      -cc.pass_config.fuse_allreduce_rms=True
      --gpu-memory-utilization 0.968
      --max-model-len 262144 --max-num-seqs 8 --max-num-batched-tokens 2048
      --max-cudagraph-capture-size 64
      --async-scheduling --enable-chunked-prefill --enable-prefix-caching
      --enable-auto-tool-choice --tool-call-parser glm47 --reasoning-parser glm45
      --default-chat-template-kwargs '{"reasoning_effort":"high"}'
      --hf-overrides '{"use_index_cache":true,"index_topk_pattern":"FFFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSSFSSS"}'
      --speculative-config '{"method":"mtp","num_speculative_tokens":5,"moe_backend":"b12x","draft_sample_method":"probabilistic"}'
volumes:
  vllm-cache:

MODEL_DIR=./SuperGLM-5.2-abliterated-hybrid docker compose up -d

First boot takes several minutes (engine init ~1–2 min + JIT compile; the healthcheck has a 600 s grace).

Verify the boot — two checks that actually matter

Sparse-MLA indexing and the KV dtype are load-bearing for long-context correctness. Check the container log:

• grep -c "skip sparse MLA indexer" must be 57 (one per sparse layer). A different count means the
  index_topk_pattern hf-override didn't apply → silent long-context corruption.
• grep "fp8_ds_mla KV cache" must be present. Missing it (i.e. BF16 KV) → garbage output on the
  B12X_MLA_SPARSE backend.

Inference

OpenAI-compatible /v1. GLM pins temperature 1.0 — do not lower it (its default inverts the usual advice):

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"GLM-5.2","messages":[{"role":"user","content":"..."}],"temperature":1.0}'

Served id is GLM-5.2; 262K context; MTP speculative decoding (5 draft tokens), chunked prefill and
prefix caching on; tool-calling (glm47 parser) and reasoning (glm45 parser) enabled.

Evaluation

Intended use & limitations

• Research use. This model has had safety refusals removed (abliterated). It will attempt
  most requests. You are responsible for how you deploy and prompt it; apply your own safety
  layer for any user-facing use.
• Quantization is lossy. NF3 3-bit experts trade some fidelity for the 4-card fit; the
  healing pass recovers most, not all, of that gap.
• License: inherits MIT from GLM-5.2. Attribution to the upstreams above is required.

Format

Standard safetensors (92 shards) + config.json carrying the hybrid metadata (hybrid_bit_map,
hybrid_scheme, per-tier layouts) + mxfp8_tier_nokvb.json. Load with the hybrid-aware vLLM image
above; a stock transformers load will not reconstruct the mixed-precision tiers.

Troubleshooting