base_model: meta-llama/Llama-3.2-3B-Instruct
tags:
- abliterated
- orthex
meta-llama/Llama-3.2-3B-Instruct (abliterated)
This model is a weight-level orthogonalized ("abliterated") version of meta-llama/Llama-3.2-3B-Instruct, produced with orthex — an implementation of the technique from Arditi et al., "Refusal in Language Models Is Mediated by a Single Direction" (NeurIPS 2024).
What was done
- Architecture adapter:
llama - Selected candidate: layer 15, site
resid_pre - Ablation targets:
embed_tokens, and every layer'sattn_outandmlp_out— orthogonalized in place in the weights (not a runtime hook; this checkpoint behaves this way standalone, with no orthex dependency at inference time)
Evaluation
| metric | pre | post | delta |
|---|---|---|---|
| refusal rate | 0.781 | 0.188 | -0.594 |
| perplexity | 18.494 | 21.785 | 3.291 |
See evaluation_report.json in this repo for the full per-prompt breakdown (refusal_samples) and the ranked candidate list considered during selection (selection_report).
Responsible use
This model has had refusal behavior removed and may comply with requests the base model would normally decline. It is intended for red-teaming, robustness research, and model-behavior analysis. Usage remains subject to the base model's original license and usage policy — this repo does not grant any additional rights beyond what meta-llama/Llama-3.2-3B-Instruct's license allows.
License
Not set automatically — inherits obligations from the base model meta-llama/Llama-3.2-3B-Instruct's license; set this field explicitly before publishing.