license: mit
base_model: jsantillana/glm-4.7-flash-abliterated
language:
- en
- es
- zh
pipeline_tag: text-generation
tags: - gguf
- llama-cpp
- abliterated
- glm
- moe
- cybersecurity
- red-teaming
- ollama
GLM-4.7-Flash-Abliterated - GGUF
Repository: jsantillana/glm-4.7-flash-abliterated-GGUF
Base Model: jsantillana/glm-4.7-flash-abliterated
GLM-4.7-Flash is a frontier 30B Mixture-of-Experts (MoE) model with Multi-Head Latent Attention (MLA) and ~3B active parameters per token, developed by Zhipu AI & Z.ai under the permissive MIT License.
This abliterated version features targeted rank-1 orthogonal subspace projection across peak refusal layers (39-46), collapsing cybersecurity false refusals by 47.6% while maintaining 98.6% language utility retention (Perplexity ratio 1.014x).
📦 Available Files & Recommended Hardware
| File | Quant Method | File Size | Recommended Hardware / VRAM | Target Use Case |
|---|---|---|---|---|
glm-4.7-flash-abliterated-Q3_K_M.gguf |
Q3_K_M | ~14.5 GB | 16 GB VRAM (RTX 4080, RTX 4070 Ti Super 16GB, Apple Silicon M-series 16GB+) | Lowest memory footprint. Enables full GPU offloading on 16GB cards with low perplexity overhead. |
glm-4.7-flash-abliterated-Q4_K_M.gguf |
Q4_K_M | ~18.5 GB | 24 GB VRAM (RTX 3090, 4090, Apple Silicon 32GB+) | Recommended for general deployment. Optimal balance of generation speed and quality retention. |
glm-4.7-flash-abliterated-Q8_0.gguf |
Q8_0 | ~32.0 GB | 48 GB VRAM / 64 GB RAM (Dual RTX 3090/4090, Apple Silicon 64GB+) | Maximum fidelity. Near-lossless 8-bit precision for high-accuracy evaluation. |
🚀 Usage Guide
1. Running with llama.cpp server
# Example for 16 GB GPU (Q3_K_M)
./llama-server \
-m glm-4.7-flash-abliterated-Q3_K_M.gguf \
-c 8192 \
--jinja \
--temp 1.0 \
--top-p 0.95 \
--repeat-penalty 1.0 \
--port 8080
# Example for 24 GB GPU (Q4_K_M)
./llama-server \
-m glm-4.7-flash-abliterated-Q4_K_M.gguf \
-c 8192 \
--jinja \
--temp 1.0 \
--top-p 0.95 \
--repeat-penalty 1.0 \
--port 8080
[!TIP]
Keep--repeat-penalty 1.0(disabled) as recommended by Z.ai for GLM-4 MoE architectures to avoid repetitive token degradation.
2. Running with Ollama
Create a Modelfile:
FROM ./glm-4.7-flash-abliterated-Q3_K_M.gguf
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.0
Then create and run:
ollama create glm4-abli -f Modelfile
ollama run glm4-abli
⚠️ Responsible Use Disclaimer
This model is intended strictly for educational purposes, academic mechanistic interpretability research, and authorized cybersecurity evaluation (Red Teaming / Penetration Testing). Do not deploy or use this model for unauthorized attacks or malicious software creation.