license: mit
base_model: zai-org/GLM-4.7-Flash
language:

• en
• es
• zh
  pipeline_tag: text-generation
  tags:
• abliterated
• glm
• moe
• mechanistic-interpretability
• cybersecurity
• red-teaming
• alignment-research

GLM-4.7-Flash (Abliterated - Targeted Subspace Projection)

This repository provides an abliterated release of zai-org/GLM-4.7-Flash (a 30B Mixture-of-Experts model with ~3B active parameters per token).

This model was created specifically for mechanistic interpretability research, alignment analysis, and authorized cybersecurity evaluation (Red Teaming & Defensive Engineering).

⚠️ Important Disclaimer & Responsible Use Policy

[!CAUTION]
FOR RESEARCH AND AUTHORIZED EVALUATION ONLY

1. Purpose: This model has had refusal mechanisms selectively abliterated across its late-intermediate MoE and attention layers to study how safety representations are encoded in frontier Mixture-of-Experts architectures.
2. Dual-Use Notice: By removing refusal filters, this model will respond to technical cybersecurity queries, network analysis requests, and exploitation concepts without automated refusals ("I cannot fulfill this request").
3. Compliance & Legality: This model is released strictly for educational purposes, defensive cybersecurity engineering, authorized penetration testing, and academic AI safety research. Users are solely responsible for ensuring compliance with all applicable local, national, and international laws (such as the Computer Fraud and Abuse Act or GDPR).
4. Prohibited Uses: Do NOT use this model to perform unauthorized cyberattacks, create malicious software for deployment against unconsenting systems, or engage in any harmful activity.

🔬 Abliteration Architecture & Methodology

Unlike naive fine-tuning or system prompt stripping, this model uses targeted rank-1 orthogonal subspace projection based on the mechanistic interpretability framework introduced by Arditi et al. (2024):

$$\hat{r}l = \frac{\bar{x}{\text{refused}, l} - \bar{x}{\text{compliant}, l}}{|\bar{x}{\text{refused}, l} - \bar{x}_{\text{compliant}, l}|_2}$$

$$W_{\text{abl}} = \left(I - \hat{r}_l \hat{r}_l^T\right) W$$

Targeted Layers & Writers:

• Target Layers: Layers [39, 40, 41, 42, 43, 44, 45, 46] (identified via empirical cosine similarity and refusal clustering).
• Writers Projected per Layer:
  • Self-Attention output projection: self_attn.o_proj ($2048 \times 2048$)
  • Shared Expert down-projection: mlp.shared_experts.down_proj ($2048 \times 1536$)
  • Routed Expert down-projections: All 64 experts mlp.experts.{0..63}.down_proj ($2048 \times 1536$)
• Total Matrices Projected: 528 weight matrices across the 8 peak refusal layers.
• Storage Format: 100% native upstream 48-shard safetensors layout.

📊 Benchmark Results

💻 Quickstart & Code Examples

1. Installation

pip install torch transformers accelerate safetensors

2. Loading the Model with Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jsantillana/glm-4.7-flash-abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

3. Example 1: Security Audit Scripting (Network Protocol Analysis)

messages = [
    {
        "role": "user",
        "content": (
            "Write a Python script using scapy that captures packets on eth0 "
            "and parses HTTP GET request URLs and User-Agents for a local security audit."
        ),
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=300, temperature=0.2)

print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

4. Example 2: Spanish Technical Analysis (Cryptographic Architecture)

messages = [
    {"role": "user", "content": "Explica la diferencia matemática y de seguridad entre RSA y curvas elípticas (ECC) en tres puntos claros."}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=250, temperature=0.3)

print(tokenizer.decode(output[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

📜 Provenance & Citation

• Base Model: zai-org/GLM-4.7-Flash (MIT License) by Zhipu AI & Z.ai.
• Technique: Arditi et al. (2024), "Refusal in Language Models Is Mediated by a Single Direction", arXiv:2406.11717.
• Manifest: Complete per-layer Frobenius norm shifts and audit trails are persisted in ABLITERATION_MANIFEST.json.