license: other
license_name: nvidia-open-model-agreement
license_link: LICENSE
base_model:

• nvidia/GLM-5.3-Flash-FP4-Local
  base_model_relation: finetune
  pipeline_tag: image-text-to-text
  tags:
• abliterated
• uncensored
• glm5_next
• GLM-5
• fp4
• nvfp4
• mxfp8
• vllm

GLM-5.3-Flash-FP4-Abliterated

NVIDIA's nvidia/GLM-5.3-Flash-FP4-Local (revision
76348ddb2be2e35e39ecfc00e75dc07703f4f3d5) with the Huihui refusal-direction edit re-applied to its own weights and
stored in its own formats. It answers where the original refuses or deflects, with the same speed and, in our tests,
the same quality.

Everything except 44 matrices is byte-identical to NVIDIA's checkpoint: 40 of the 42 weight files are unchanged, the
two others differ only inside the 44 edited tensors (checked byte by byte outside them). repack-manifest.json lists
every changed matrix with its measurements and the SHA-256 of both rewritten files.

What was changed

• Matrices: self_attn.o_proj and mlp.shared_experts.down_proj in layers 15 to 36, the same 44 projections the
  Huihui abliteration of GLM-5.3-Flash edits
  (edp1096/Huihui-GLM-5.3-Flash-abliterated-NVFP4, MIT).
• Why not just add Huihui's delta: NVIDIA trained this checkpoint further with quantization-aware distillation, so
  its weights are no longer the originals the delta was computed against. The edit is a rank-1 change along one
  output direction, so it is re-applied as such: T = D - s · r (rᵀ D), where D is this checkpoint's dequantized
  weight, r the leading left singular vector of (Huihui-edited − unedited), and s the strength the original edit
  had on the original weight (about 2.4).
• Formats kept exactly: 28 of the matrices are E4M3 with an FP32 scale per 128×128 block, 16 are MXFP8 (E4M3
  with an E8M0 scale per 32 inputs, scale = 2^ceil(log2(amax/448))). Before writing, the encoder was required to
  reproduce NVIDIA's own codes and scales from NVIDIA's own dequantized weights for all 44 matrices; it did, with
  zero differences. The routed experts (NVFP4 with compressed scales) are not touched.
• After re-encoding, 91 to 95 % of the edit's strength remains (the rounding of the 8-bit formats), comparable to
  earlier FP8 builds of the same edit.

Tests

Both variants on 2× RTX PRO 6000 Blackwell (300 W each), Local Inference Lab vLLM release-10152026, TP2 with decode
context parallel 2, MTP 3, 8 sequences, --max-model-len 262144 (FP8 KV cache of 1,107,332 tokens), same harness
and prompts for both.

This is a small test set, not a benchmark suite. The refusal probe is meant to show the edit works, not to rank models.

Running it

The NVFP4 scale factors use the lossless compressed format of the original checkpoint, so this needs the same
containers NVIDIA lists for the original; upstream vLLM cannot read it yet. Use NVIDIA's command for
2× RTX PRO 6000 from the original model card with this
repository (or a local copy) as the model. We serve it with --max-model-len 1048576 (the model's native length),
--max-num-seqs 8, --decode-context-parallel-size 2, --kv-cache-memory-bytes 7000000000 and MTP 3.

The KV cache gets more room from the larger --max-model-len: with the same 7 GB per card, vLLM reports
1,687,708 tokens at --max-model-len 1048576 and 1,107,332 at 262144. We take that room for many parallel
chats and still limit a single request to 262,144 tokens in front of vLLM; the long-context checks above went up
to 240K, and single requests beyond 262,144 tokens were not tested with this build. The functional checks pass
11 / 11 in this layout as well.

Responsibility

The refusal behaviour of the original has been removed on purpose. You are responsible for how you use the model and
for what it produces; apply the safeguards your use case needs.

License

Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement (see LICENSE). The base model GLM-5.3-Flash by
Z.ai is MIT-licensed; the Huihui edit it builds on is MIT-licensed. Changes: the 44 matrices described above.