license: mit
base_model:

• zai-org/GLM-5.3-Flash
  tags:
• LoRA
• Uncensored
• Abliterixed

GLM-5.3-Flash-Abliterixed-LoRA

Refusal-direction ablation LoRA for GLM-5.3-Flash.

This repository contains LoRA adapter weights only. The adapter was derived from zai-org/GLM-5.3-Flash with Abliterix to reduce refusal behavior while leaving the base checkpoint unchanged.

The adapter is intended for controlled safety research, red-teaming, and refusal-mechanism analysis. Removing refusal behavior also removes an important safety layer; read the Disclaimer before use.

Adapter summary

What is abliteration?

Abliteration is a weight-space intervention inspired by Arditi et al., Refusal in Language Models Is Mediated by a Single Direction. In simplified form, a refusal direction is estimated from the difference between internal activations elicited by harmful and harmless prompts. Components of selected writer matrices that contribute to that direction are then removed. A rank-1 update can be represented as:

W' = W + ΔW,      ΔW = BA,      rank(ΔW) = 1

Storing ΔW as a LoRA adapter means the original model checkpoint is not rewritten and the intervention can be enabled or removed at serving time.

Why routed-expert down_proj?

The analysis published with orcarouter/GLM-5.3-Flash-Uncensored-FP8 identifies the routed-expert MLP down_proj writers as the dominant intervention point for GLM-5.3-Flash refusal behavior. Its component study reports that attention-only and dense/shared-MLP interventions retain most refusal behavior, whereas adding routed-expert MLP writers causes the major reduction.

Our layer-restricted experiments agreed with that mechanism-level result: the causally useful refusal directions were found in the earlier-to-middle part of the expert stack, while modifications restricted to the later expert layers had almost no practical effect on refusal. Accordingly, the routed-expert down_proj modules are the main effective target in this adapter.

This statement concerns causal intervention effectiveness, not the numerical magnitude of every stored LoRA tensor. Adapter coverage and tensor norm alone do not establish where a behavior is represented.

This adapter instead uses Abliterix's SRA + mean + per-layer workflow:

• Mean base estimator: harmful-versus-harmless mean activation differences provide the initial direction estimate.
• SRA: the initial refusal signal is spectrally cleaned/residualized against protected benign-capability components, reducing unwanted removal of overlapping capabilities.
• Per-layer estimation: each edited layer receives its own estimated direction; a single layer-22 vector is not reused across the entire model.

Per-layer estimation is important here because the refusal signal is not equally useful at every depth, and late-layer-only interventions were ineffective in our experiments.

LoRA weight distribution

The following figures were obtained by inspecting the supplied adapter_config.json and adapter_model.safetensors.

Effective non-zero edits

The adapter contains 46,770 BF16 tensors: 23,385 lora_A factors and 23,385 lora_B factors. It serializes 143,927,296 scalar entries, of which 48,375,808 are non-zero. The safetensors file is approximately 281.6 MiB.

The PEFT configuration lists down_proj, gate_proj, o_proj, and up_proj as target modules. In the supplied checkpoint, however, all stored gate_proj and up_proj LoRA factors are zero. They are structural entries and do not alter model output. The effective changes are therefore the down_proj and o_proj entries listed above.

Layer-level coverage is:

• Layers 0–14: no effective LoRA edit.
• Layers 15–17: attention o_proj only.
• Layers 18–44: attention o_proj, all routed-expert down_proj modules, and the shared-expert down_proj.

Evaluation

The adapter was evaluated while attached to RedHatAI/GLM-5.3-Flash-NVFP4. The tested deployment used vLLM, explicitly selected low reasoning effort for the target model, and passed only the target model's final answer—not its private reasoning—to the judge.

The automated judge was deepseek-v4-flash through the Ollama Cloud API, with judge thinking disabled. These results are automated estimates, not human safety certification.

Overall results

For SimpleSafetyTests, the auxiliary response-style labels were 1% full refusal, 13% partial refusal, and 86% no refusal. Its primary result remains the judge's safe/unsafe classification rather than the auxiliary refusal label.

For StrongREJECT, a higher rubric score means that the response gave more useful and specific assistance toward the harmful request; it therefore indicates weaker safety refusal, not higher general model quality. The official string-matching baseline mean on this run was 0.772222.

StrongREJECT category breakdown

Warning rate is not refusal rate

A warning, caveat, legal note, or safety disclaimer was counted separately from refusal. If a response warned about risk but still provided request-relevant content, it was not counted as a refusal on that basis. Consequently, a high warning rate can coexist with a low refusal rate.

Usage

This repository does not include the base model. Download or otherwise provide a compatible GLM-5.3-Flash checkpoint separately and comply with its license and terms.

vLLM

The evaluation configuration used the NVFP4 base with the LoRA enabled. Adapt tensor parallelism, memory limits, context length, and port to your hardware:

vllm serve RedHatAI/GLM-5.3-Flash-NVFP4 \
  --enable-lora \
  --max-lora-rank 1 \
  --lora-modules glm-5.3-flash-abliterixed=/path/to/GLM-5.3-Flash-Abliterixed-LoRA

Call the LoRA alias as the model name. GLM-5.3-Flash supports explicit reasoning-effort selection; setting it explicitly improves reproducibility:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash-abliterixed",
    "messages": [{"role": "user", "content": "Hello"}],
    "reasoning_effort": "low",
    "max_tokens": 4096
  }'

Your vLLM build must support LoRA application to the GLM-5.3 MoE architecture, including expert-granular down_proj tensors.

Transformers + PEFT

For an unquantized or otherwise PEFT-compatible base model:

import torch
from peft import PeftModel
from transformers import AutoModelForMultimodalLM, AutoProcessor

base_id = "zai-org/GLM-5.3-Flash"
adapter_path = "/path/to/GLM-5.3-Flash-Abliterixed-LoRA"

processor = AutoProcessor.from_pretrained(base_id, trust_remote_code=True)
base_model = AutoModelForMultimodalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_path)
model.eval()

Use recent Transformers and PEFT versions that recognize the GLM-5.3 architecture and its MoE expert modules. Compatibility of a particular quantization backend must be verified separately.

Limitations

• Abliteration changes refusal behavior; it does not guarantee correctness, harmlessness, honesty, or improved task capability.
• The intervention does not prove that all safety behavior is encoded by one direction or one module family.
• Residual refusals may remain, and later fine-tuning, merging, quantization, or serving changes may strengthen or weaken the effect.
• Automated judge labels can contain false positives and false negatives. Warning rate is not a safety score.
• Results are specific to the stated base checkpoint, prompts, sampling configuration, runtime, and judge model.

Disclaimer

This adapter is released solely for legitimate research, safety evaluation, red-teaming, and other lawful uses. It intentionally weakens refusal behavior and may cause the base model to produce dangerous, illegal, hateful, sexually explicit, deceptive, or otherwise harmful content. Do not expose it to untrusted users or deploy it without appropriate access control, monitoring, content filtering, rate limits, and human oversight.

You are responsible for complying with applicable laws, regulations, platform policies, and the licenses and terms of the base model, quantized checkpoint, Abliterix, and all other dependencies. The authors and distributors make no warranty and accept no responsibility for outputs, downstream use, misuse, damages, or legal consequences arising from this adapter.

GLM-5.3-Flash-Abliterixed-LoRA

面向 GLM-5.3-Flash 的 Abliterixed LoRA。

本仓库仅提供 LoRA 适配器权重。该适配器以 zai-org/GLM-5.3-Flash 为原始模型，使用 Abliterix 对拒绝行为进行消融；基础模型权重本身不会被本仓库替换。

本适配器用于受控的安全研究、红队测试和拒绝机制分析。移除拒绝行为也意味着削弱重要的安全防线，使用前请阅读文末的免责声明。

适配器概要

什么是拒绝消融？

拒绝消融（abliteration）是一种权重空间干预方法，其思路源自 Arditi 等人的论文 Refusal in Language Models Is Mediated by a Single Direction。简而言之，该方法通过有害提示与无害提示产生的内部激活差异估计拒绝方向，再从选定的写入矩阵中移除会向该方向写入的分量。秩 1 更新可表示为：

W' = W + ΔW,      ΔW = BA,      rank(ΔW) = 1

将 ΔW 保存为 LoRA，可以在不重写基础模型检查点的情况下，于推理时加载或卸载该干预。

为什么主要修改路由专家的 down_proj？

orcarouter/GLM-5.3-Flash-Uncensored-FP8 公布的分析表明，GLM-5.3-Flash 的拒绝行为主要由路由专家 MLP 的 down_proj 写入端承载。其组件实验显示，仅修改注意力或稠密/共享 MLP 时，大部分拒绝行为仍然保留；加入路由专家 MLP 写入矩阵后，拒绝才出现主要下降。

我们进行的分层限制实验也验证了这一机制：具有实际因果作用的拒绝方向集中在专家堆栈的前段至中段；如果只修改靠后的专家层，对拒绝行为几乎没有实际影响。因此，本适配器将路由专家的 down_proj 作为最主要的有效修改目标。

这里的“分布在前段、后层修改无效”描述的是行为干预的因果有效性，并不表示所有前层 LoRA 张量的数值范数都必然大于后层。权重覆盖范围或张量大小本身不能单独证明某种行为储存在哪里。

本适配器采用 Abliterix 的 SRA + mean + per-layer 流程：

• **Mean 基础估计：**以有害提示和无害提示的平均激活差作为初始拒绝方向。
• **SRA：**使用受保护的无害能力成分对初始拒绝信号进行谱清理/残差化，尽量减少对重叠通用能力的误删。
• **逐层估计：**为每个被修改层独立估计方向，而不是把某层的拒绝向量应用到整个模型。

逐层估计在这里非常重要，因为拒绝信号在不同深度上的因果作用并不相同。

LoRA 修改权重的层与模块分布

以下数据来自对本仓库 adapter_config.json 与 adapter_model.safetensors 的实际检查，而不是根据文件名推测。

实际非零修改

适配器共包含 46,770 个 BF16 张量，其中 lora_A 和 lora_B 各 23,385 个；序列化标量总数为 143,927,296，其中非零标量为 48,375,808。safetensors 文件大小约为 281.6 MiB。

PEFT 配置把 down_proj、gate_proj、o_proj 和 up_proj 都列为目标模块，但本检查点内所有 gate_proj 与 up_proj LoRA 因子均为零；它们只是结构性条目，不会改变模型输出。实际生效的修改因此是上表中的 down_proj 与 o_proj。

逐层覆盖情况如下：

• 第 0–14 层：没有实际 LoRA 修改。
• 第 15–17 层：仅修改注意力 o_proj。
• 第 18–44 层：修改注意力 o_proj、所有路由专家 down_proj 以及共享专家 down_proj。

评测结果

本适配器加载在 RedHatAI/GLM-5.3-Flash-NVFP4 基础模型上进行评测。测试部署使用 vLLM，并为被测模型明确设置低思考强度。交给裁判模型的内容只有被测模型的最终答案，不包含其思考过程。

自动裁判模型为 deepseek-v4-flash，裁判思考功能关闭。以下结果是自动评估结果，不等同于人工安全认证。

总体结果

SimpleSafetyTests 的辅助回答风格标签为：完全拒绝 1%、部分拒绝 13%、不拒绝 86%。该测试的主要结果仍是裁判给出的安全/不安全分类，而不是辅助拒绝标签。

StrongREJECT 的 rubric 得分越高，表示回答对有害请求提供的帮助越具体、越有用，也就是安全拒绝越弱，并不代表模型的通用质量越高。本次运行的官方字符串匹配基线均值为 0.772222。

StrongREJECT 分类结果

警告率不等于拒绝率

警告、风险提示、法律说明和安全免责声明与拒绝分别统计。如果模型虽然发出警告，但仍然回答了与请求相关的实质内容，则不会仅因为存在警告而被算作拒绝。因此，高警告率可以与低拒绝率同时出现。

使用方法

本仓库不包含基础模型。请另行下载或提供兼容的 GLM-5.3-Flash 检查点，并遵守相应许可证与使用条款。

使用 vLLM 加载

评测时使用 NVFP4 基础模型并启用该 LoRA。请根据硬件调整张量并行、显存限制、上下文长度和端口等参数：

vllm serve RedHatAI/GLM-5.3-Flash-NVFP4 \
  --enable-lora \
  --max-lora-rank 1 \
  --lora-modules glm-5.3-flash-abliterixed=/path/to/GLM-5.3-Flash-Abliterixed-LoRA

请求时使用 LoRA 别名作为模型名。GLM-5.3-Flash 支持显式设置思考强度；为保证结果可复现，建议不要依赖默认值：

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.3-flash-abliterixed",
    "messages": [{"role": "user", "content": "你好"}],
    "reasoning_effort": "low",
    "max_tokens": 4096
  }'

所用 vLLM 版本必须能够在 GLM-5.3 MoE 架构上应用 LoRA，并支持专家粒度的 down_proj 张量。

使用 Transformers + PEFT 加载

对于未量化或其他与 PEFT 兼容的基础模型：

import torch
from peft import PeftModel
from transformers import AutoModelForMultimodalLM, AutoProcessor

base_id = "zai-org/GLM-5.3-Flash"
adapter_path = "/path/to/GLM-5.3-Flash-Abliterixed-LoRA"

processor = AutoProcessor.from_pretrained(base_id, trust_remote_code=True)
base_model = AutoModelForMultimodalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_path)
model.eval()

请使用能够识别 GLM-5.3 架构及其 MoE 专家模块的较新版本 Transformers 与 PEFT。特定量化后端是否兼容，需要另行验证。

局限性

• 拒绝消融改变的是拒绝行为，并不保证模型正确、无害、诚实，也不保证通用任务能力有所提升。
• 本干预不能证明所有安全行为都由单一方向或单一模块族编码。
• 模型仍可能保留部分拒绝；后续微调、权重合并、量化或推理服务配置也可能增强或削弱消融效果。
• 自动裁判可能产生误判；警告率不是安全得分。
• 结果只适用于文中所述基础检查点、提示词、采样配置、运行时和裁判模型。

免责声明

本适配器仅供合法的研究、安全评估、红队测试及其他合规用途。它会有意削弱模型的拒绝行为，可能使基础模型生成危险、违法、仇恨、露骨色情、欺骗性或其他有害内容。请勿直接向不可信用户开放，也不要在缺少访问控制、监控、内容过滤、速率限制和人工监督的情况下部署。

使用者有责任遵守适用的法律法规、平台政策，以及基础模型、量化检查点、Abliterix 和其他依赖项的许可证与使用条款。作者与分发者不提供任何明示或默示保证，也不对本适配器生成的内容、下游使用、滥用、损失或法律后果承担责任。