license: other
license_name: nvidia-open-model-agreement
base_model:
- nvidia/GLM-5.3-Flash-FP4-Local
- huihui-ai/GLM-5.3-Flash-abliterated-GGUF
base_model_relation: finetune
pipeline_tag: image-text-to-text
library_name: Model Optimizer
tags: - glm5_next
- modelopt
- nvfp4
- mxfp8
- abliterated
Huihui-GLM-5.3-Flash-abliterated-FP4-Local
Unofficial NVIDIA FP4-Local derivative with Huihui Q8 deltas applied.
Sources
| Role | Source | Pinned revision |
|---|---|---|
| Base FP4-Local checkpoint | nvidia/GLM-5.3-Flash-FP4-Local | 76348ddb2be2e35e39ecfc00e75dc07703f4f3d5 |
| Abliteration donor (Q8) | huihui-ai/Huihui-GLM-5.3-Flash-abliterated-GGUF | 50d39f500cbd1d0478295da89d417607521d378b |
| Delta reference (Q8) | unsloth/GLM-5.3-Flash-GGUF | 621d456e93e926e4b52f85cff5f634358c1828f9 |
Applied delta: DQ(Huihui Q8) - DQ(Unsloth Q8). The conversion used these exact revisions.
Changes
- Updated 44 attention-output/shared-expert-down matrices in layers 15–36 and 22 scale tensors
- Preserved all other tensors, including routed experts, CSF scales, MTP, vision, and tokenizer files
- Requantized in the existing FP8 formats; no additional QAD training or activation calibration
Runtime
Requires two DGX Sparks (TP2) and B12X vLLM with CSF support.
See runtime-configuration.json for exact tested settings.
PYTORCH_ALLOC_CONF=backend:native,garbage_collection_threshold:0.7
Cold-start loading caused memory pressure and some host swapping.
Validation
- All 42 weight files verified; 8/8 smoke checks passed, including code, tools, vision, and long-context retrieval
- Longest tested input: 64,481 tokens. Configured limit: 1,048,576; actual 1M requests were not tested
- Huihui BF16 equivalence and preservation of upstream benchmark scores are not guaranteed
Results · Sources, changes, and hashes · NVIDIA model card and license