base_model: Qwen/Qwen3-32B
license: apache-2.0
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- abliterated
- uncensored
- ulre
- qwen3
language: - en
Qwen3-32B-ULRE (abliterated, MLX 4-bit)
An abliterated (refusal-reduced) build of Qwen/Qwen3-32B,
4-bit MLX, produced with ULRE (Universal Layerwise Rank-1 steering) — a per-layer residual-stream
steering edit baked into the attention output projection.
Qwen3-32B is a modern (2025) reasoning model whose safety resisted classic single-direction
abliteration (0% clean compliance on a deflection-aware harmful-prompt eval). ULRE de-refuses it
cleanly and preserves/improves capability.
Results
Judged by an independent local LLM judge on 100 held adversarial prompts @512 tokens (thinking ON),
scoring 0=refuse … 1=clean compliance … 4-5=strong steering/propaganda.
| base Qwen3-32B | this model (ULRE) | |
|---|---|---|
| clean compliance | 0 / 100 | 68 / 100 |
| refuse | 86–97 | 18 |
| strong-steer (4-5) | — | 4 |
Capability gates (vs base Qwen3-32B 4-bit) — improved across the board (the base over-refuses
on some capability prompts; abliteration recovers them):
| gate | base | this model |
|---|---|---|
| tool-call validity | 0.95 | 0.97 |
| math (GSM8K-lite) | 0.88 | 0.94 |
| code (HumanEval-lite) | 0.325 | 0.50 |
Method (ULRE)
Modern refusal behaves like a routed control circuit, not a single residual feature. ULRE subtractsalpha * u_l (the layer-l harmful−harmless activation mean-difference direction) from the output of
a band of decoder layers (here o_proj on layers 28–43, alpha=16). This is baked statically into
each window layer's o_proj as a bias term o_proj.bias = -alpha * u_l. See the project'sdocs/ULRE_DESIGN.md.
⚠️ Loading — requires a one-line mlx-lm patch
Because Qwen3's attention output projection has no bias by default, mlx-lm must be told to build ano_proj bias. Add an optional flag to mlx_lm/models/qwen3.py (backwards-compatible — base models
default to False):
# in class ModelArgs:
o_proj_bias: bool = False
# in class Attention.__init__:
self.o_proj = nn.Linear(n_heads * head_dim, dim, bias=getattr(args, "o_proj_bias", False))
The model's config.json sets "o_proj_bias": true. (A PR to upstream this optional flag is in
progress; once merged this patch is unnecessary.) Then load normally:
from mlx_lm import load, generate
model, tok = load("gregfrank/Qwen3-32B-ULRE-abliterated")
print(generate(model, tok, prompt=tok.apply_chat_template(
[{"role": "user", "content": "Hello"}], tokenize=False, add_generation_prompt=True),
max_tokens=256))
Intended use & safety
Research artifact for studying refusal mechanisms and safety-tuning robustness. It will comply with
requests a stock model refuses. Use responsibly and in accordance with the Apache-2.0 license and
applicable law.