license: apache-2.0
base_model: SupraLabs/Supra-50M-Instruct
tags:

• mlx
• abliteration
• uncensored
• apple-silicon
• llama
  language:
• en

Supra-50M-Uncensored (MLX)

Uncensored version of SupraLabs/Supra-50M-Instruct, produced via single-direction abliteration (Arditi et al., 2024 — arXiv:2406.11717) on Apple Silicon using MLX.

⚠️ Research use only. This model will produce harmful content on request. Do not deploy in products or use to cause harm. You are responsible for what you generate.

What changed

Aligned language models encode the decision to refuse in a single direction in the residual stream. Abliteration identifies that direction by contrasting activations on harmful vs. harmless prompts, then orthogonalizes it out of every residual-stream weight in the model — permanently, with no fine-tuning required.

For this model:

• Direction source: layer 2 activations (selected by a full-model sweep across layers 2–10)
• Weights edited: embed_tokens, all self_attn.o_proj, all mlp.down_proj
• Base dtype preserved: BFloat16
• Architecture: Llama (plain transformer, 12 layers, 512 hidden, 50M params)

The base model was SFT'd on Alpaca-cleaned (not RLHF safety-trained), so its refusal rate was already low. Abliteration removes the residual 4 pp of refusing behavior without affecting harmless instruction following.

Benchmark results

Evaluated on Apple Silicon (MLX, greedy decoding, temp=0). Refusal detection via keyword matching on the final response (uncensor/core/refusal.py).

The single remaining refusal on AdvBench-100 is a content-style edge case where the model's phrasing happens to match a refusal keyword despite generating compliant content; general capability and instruction following are fully preserved.

Usage

from mlx_lm import load, generate

model, tokenizer = load("sahilchachra/Supra-50M-Uncensored")

messages = [{"role": "user", "content": "Your prompt here"}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False
)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(response)

The tokenizer uses an Alpaca-format chat template (matching the SFT training format):

<s>Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
{your prompt}

### Response:

Model card for the base model

See SupraLabs/Supra-50M-Instruct for architecture details, training data, and intended use of the original model.

Technique

Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024)

Abliteration toolkit: github.com/sahilchachra/uncensor-llms

Published by

sahilchachra