language:
- en
license: mit
library_name: transformers
tags: - flux
- flux.2-klein
- abliteration
- text-encoder
- qwen3
- diffusion
base_model: black-forest-labs/FLUX.2-klein-base-4B
Flux.2-klein-4B Abliterated Text Encoder
Repository:
PinoCookie/Flux.2-klein-4B-abliterated-text-encoder
Base model:black-forest-labs/FLUX.2-klein-base-4B
Method: Multi-pass weight orthogonalization (α=0.5 × 3)
Target: Text encoder only (Qwen3, 36 layers, 4B params)
Summary
Abliterated text encoder for FLUX.2-klein-4B. The DiT (image generator) has no guardrails — all safety filtering lives in the text encoder. Replace the text encoder with this abliterated version and ALL prompts pass through to the DiT unhindered.
Method
Probe
- Harvested residual stream activations from all 36 layers on 20 harmful + 20 harmless prompts
- Computed per-layer refusal direction via difference-in-means
- Refusal spike confirmed at layers 32-35 (separation scores 160-232, matching ponpoke's analysis)
| Layer | Separation |
|---|---|
| 35 | 232.4 |
| 34 | 214.4 |
| 33 | 179.3 |
| 32 | 160.3 |
| 31 | 144.4 |
| ... | decreasing linearly |
Excise
- Target: 10 layers (26-35)
- Weights:
self_attn.o_proj+mlp.down_proj(attention output + MLP output) - Passes: 3 passes at α=0.5 each (cumulative α=1.5 applied incrementally)
Verification
| Metric | Value | Interpretation |
|---|---|---|
| Harmless cos sim (original vs ablated) | 0.9965 | Capability preserved ✓ |
| Harmful cos sim (original vs ablated) | 0.9080 | Refusal filter diverged ✓ |
Usage
import torch
from diffusers import Flux2KleinPipeline
# Load base model
pipe = Flux2KleinPipeline.from_pretrained(
"black-forest-labs/FLUX.2-klein-base-4B",
torch_dtype=torch.bfloat16,
trust_remote_code=True
).to("cuda")
# Replace text encoder with abliterated version
state_dict = torch.load("text_encoder.pt", map_location="cuda")
pipe.text_encoder.load_state_dict(state_dict)
# Generate anything — no refusal
image = pipe("a realistic depiction of...").images[0]
image.save("output.png")
Credits
- ponpoke — Original discovery that Flux.2 Klein's DiT has no guardrails and safety lives in the text encoder. Published
ponpoke/flux2-klein-4b-uncensored-text-encoder. - Alosh Denny — Pioneered diffusion model abliteration (Flux.1 Dev v1/v2).
- Andy Arditi et al. — "Refusal in LLMs is Mediated by a Single Direction" (foundational work).