license: mit
base_model: zai-org/GLM-5.3-Flash
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: transformers
language:

• en
• zh
  tags:
• abliterated
• uncensored
• glm
• glm5
• glm-5.3-flash
• glm5-next
• moe
• nvfp4
• fp4
• compressed-tensors
• vllm
• vision-language
• function-calling
• reasoning
• mtp
• ai-red-team
• red-teaming

GLM-5.3-Flash-Uncensored-NVFP4

NVFP4 (4-bit) weight quantization of the abliterated (refusal-removed) GLM-5.3-Flash — experts-only FP4, served on vLLM

One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.

Website · Model Catalog · GitHub · Discord · X

NVFP4 weight quantization of the abliterated (refusal-removed) build of Z.ai's
GLM-5.3-Flash — a 320B / 18B-active
Mixture-of-Experts model (glm5_next, 288 routed experts top-8 + 1 shared) with hybrid linear +
sparse attention, 4-wide Manifold-Constrained Hyper-Connections (mHC), a native vision + video
tower, an MTP speculative head, and a 1M-token context. The routed-expert weights are
quantized to NVFP4 (4-bit, NVIDIA FP4 E2M1, group-16 + FP8-E4M3 group scales + FP32 global scale) —
that is 97% of the parameters — while everything else is kept BF16, cutting the checkpoint from
306 GiB (block-FP8) to ~177 GiB with no measurable loss outside the experts. Browse all models
in the OrcaRouter Model Catalog.

⚠️ Disclaimer — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or
illegal requests the original GLM-5.3-Flash would refuse — it has no meaningful built-in guardrails.
Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study,
red-teaming, and robustness evaluation. You assume full responsibility for how you use it and
everything it generates; add your own safety, moderation, and abuse-prevention layers before any
deployment. Use must comply with the MIT License inherited from the
base model and all applicable law. The authors accept no liability for misuse, and its outputs do
not reflect the views of the uploaders or of Z.ai / Zhipu AI.

Model details

What's quantized

• Weight-only, data-free. Expert weights are derived directly from the source checkpoint
  (symmetric FP4, per-group absmax); activations are quantized dynamically at runtime — no static
  calibration corpus. The abliteration is baked into the weights, so refusal-removal is preserved.
• F32 aux kept F32. Some public NVFP4 builds downcast the hyper-connection / SSM / router-bias
  aux tensors to BF16; those tensors are precision-sensitive (bf16 error is amplified through the SSM
  recurrence and the mHC residual), so this build keeps them FP32 at ~zero size cost.
• MTP block dropped by default (runtimes drop it on load anyway); speculative decoding is off. Ping
  us if you need the MTP-retained variant.
• KV cache is not quantized (BF16 at runtime).

Recipe note. This mirrors the target split of the reference build
RedHatAI/GLM-5.3-Flash-NVFP4 — experts-only
NVFP4 — and is byte-format-identical to it (same weight_packed / weight_scale /
weight_global_scale layout), so it loads through the same vLLM path. It differs in two ways: it is
built from the abliterated checkpoint, and it keeps the F32 aux tensors at F32.

Requirements

• A vLLM build with glm5_next (GLM-5.3-Flash) support. This is a brand-new architecture and is not
  in stock vLLM yet (tracked in vllm#53906). Use the
  official image vllm/vllm-openai:glm53-flash-x86_64-cu129 (or the ROCm/arm/cu130 variants), or a build
  from that PR. transformers>=5.16 provides the reference arch.
• GPU. Verified serving on 8×H100 (Hopper) via the official image — vLLM runs the compressed-tensors
  NVFP4 path on Hopper. A Blackwell GPU (B100/B200/GB200) additionally has native FP4 tensor cores for
  best throughput. Plan for ~177 GiB of weights plus KV cache (e.g. 8×H100/H200 with tensor parallelism).
• Hopper does not support FP8 KV cache for this model — omit --kv-cache-dtype fp8 there.

Usage — self-host with vLLM (OpenAI-compatible)

docker run -d --name glm53-nvfp4 --gpus all --ipc=host -p 8000:8000 \
  -v /path/to/GLM-5.3-Flash-Uncensored-NVFP4:/model:ro \
  -e VLLM_SSM_CONV_STATE_LAYOUT=DS -e VLLM_KV_CACHE_LAYOUT=HND \
  vllm/vllm-openai:glm53-flash-x86_64-cu129 \
  --model /model --served-model-name GLM-5.3-Flash-Uncensored-NVFP4 \
  --tensor-parallel-size 8 --max-model-len 8192 --enforce-eager --trust-remote-code

Thinking control

GLM-5.3-Flash has no enable_thinking toggle — its chat template always opens a <think> block.
Control the budget with reasoning_effort (low / high / max; defaults to max), and pass
clear_thinking: true for chat:

client.chat.completions.create(
    model="GLM-5.3-Flash-Uncensored-NVFP4",
    messages=[{"role": "user", "content": "..."}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "low", "clear_thinking": True}},
)

Give generation enough budget to reach </think>, or replies get truncated inside the scratchpad.

Evaluation

Measured on this build vs its block-FP8 abliterated reference. All numbers are from these exact
weights, not inherited from the base card.

Weight-space fidelity (NVFP4 experts vs FP8)

Sampled across all 42 quantized expert layers (gate + down), NVFP4 round-trip vs the source:

9.18% is the intrinsic floor of 4-bit FP4 for these narrow experts (moe_intermediate = 2048) and is
uniform across layers — no per-layer degradation.

Perplexity / KLD / Top-1 vs the FP8 reference (wikitext-2)

Forward-pass measurement of the NVFP4 recipe against the FP8-dequant reference (24,564 predicted tokens):

KLD 0.073 and Top-1 91.7% place this squarely in the 4-bit tier (for comparison, a 3-bit build of a
sibling model lands at KLD ~0.17 / Top-1 ~90%). A direct vLLM load of this build measured PPL within
the same band (a real-kernel W4A4 forward reads a few points higher than the weight-only forward, as
expected for dynamic activations).

Uncensoring retained after quantization

Held-out JailbreakBench test split (64 harmful + 64 benign), served with vLLM on this exact build,
greedy, reasoning_effort=low, <think> stripped before a rule-based classifier:

Harmful refusal stays in the single-digit-to-teens band — down from ~90% on the base model — and
benign over-refusal is zero. The few extra points over the FP8 abliteration are the expected residual:
the refusal direction lives almost entirely in the routed-expert down_proj matrices, and 4-bit
quantization of exactly those weights leaves a little more of it behind than FP8 does. The build answers
harmful requests directly (82.8% compliance) while often wrapping the answer in a disclaimer (a caveat,
not a refusal). Refusal is reduced, not removed — some content categories the direction does not reach
still refuse (see the base abliteration card).

Method note. The refusal metric is a rule-based opening-phrase classifier (bilingual EN/ZH),
indicative rather than an LLM-judge / publication-grade number. Evaluate rigorously for your own use case.

Fine-tuning & re-quantization

• Loads through any vLLM build with glm5_next + compressed-tensors NVFP4 support.
• Abliteration is a weight edit, not data-level unlearning: fine-tuning on refusal-heavy / safety data
  can partially re-introduce refusals; neutral / task data preserves the uncensored behaviour.
• For a higher-fidelity build, keep the expert down_proj at FP8 (mixed-precision, larger); for a smaller
  one, a 3-bit MLX build trades ~2× the KLD for ~20 GiB.

Bias, risks, and limitations

• Safety guardrails removed — will produce harmful, biased, or offensive content on request.
• Inherits any biases and limitations of the base GLM-5.3-Flash.
• 4-bit expert quantization adds a small quality trade-off vs the FP8 source (see Evaluation) and leaves a
  few extra points of residual refusal.
• Capability is expected to track the base within measurement noise; the numbers above are on sampled
  corpora, not a full harness run.

License

MIT, inherited from the base model
zai-org/GLM-5.3-Flash. Abliteration and quantization do
not change the underlying license obligations.