M1 - Direct removal

What M1 is

M1 is the original technique. Find the single direction in the model's activation space that mediates refusal, then permanently edit the weights so the model can no longer represent that direction. Nothing is added back; the model ships as-modified. This is the method described in the Arditi paper, and every other method in the M1-M8 catalog is either a wrapper around it (Heretic), a refinement of it (M3), or a way to fix its collateral damage (M4). If you understand M1 you understand the field.

This article is the practitioner reference: what the procedure actually does at the level of code, which tools to use, what choices you have to make, how to tell whether the result is good. For the historical and conceptual account of the same technique - who wrote the paper, what it exactly claims, what has been contested - see the Arditi method article. Both cover M1 by design.

What you need before starting

• A chat or instruct model. M1 works on instruction-tuned models; base models have no refusal behavior to remove. For a first run: Qwen/Qwen3-1.7B, google/gemma-2-2b-it, or meta-llama/Meta-Llama-3-8B-Instruct.
• Two prompt sets. A harmful set and a harmless set of comparable length and structure. The canonical harmless set is tatsu-lab/alpaca (repackaged as mlabonne/harmless_alpaca). The canonical harmful set is the harmful_behaviors split of AdvBench (repackaged as mlabonne/harmful_behaviors, 520 prompts). Fetch these from the Hub, do not paste examples.
• Disk: 2-3x the FP16 model size for a comfortable margin. VRAM: a 24 GB card handles up to 13B comfortably at full precision, more in 4-bit. Time: 10-30 minutes for a small model. Cost: under $1 on a rented 3090 or 4090.

The three tool lineages

Three families of implementations descend from the paper, and the choice between them shapes everything that follows.

Arditi's refusal_direction is the reference research code, released with the paper. It uses TransformerLens, the interpretability library maintained originally by Neel Nanda and Joseph Bloom. The code is exact and well-tested; it is also constrained to whatever TransformerLens supports, which historically has meant a lag of weeks to months on new architectures.

FailSpy's abliterator library is what popularized the technique in mid-2024 and is where the word abliteration was coined. It sits at roughly 640 GitHub stars, is TransformerLens-based, and is effectively in maintenance mode (no formal releases in the recent period). It remains historically important as the artifact that carried the paper's technique to the community.

Sumandora's remove-refusals-with-transformers is the community default for a from-scratch, dependency-light run in late 2026. Two thousand stars, 305 forks, and - critically - no TransformerLens dependency. It uses plain Hugging Face Transformers and PyTorch hooks, so it supports any architecture Transformers supports on the day of release. A large share of community abliterations descend from this codebase.

Labonne's Colab notebook remains the most-referenced teaching path. It is TransformerLens-based, and several commenters report it silently failing on newer small models (e.g. Llama-3.2-3B) as the ecosystem drifts past TransformerLens support. Labonne now points people toward his AutoAbliteration successor.

$ git clone https://github.com/Sumandora/\
    remove-refusals-with-transformers
$ cd remove-refusals-with-transformers
$ pip install -r requirements.txt

# core deps:
#   transformers >= 4.40
#   torch        >= 2.2
#   accelerate

# you also need two prompt files
# (one instruction per line):
#   harmful.txt   ← from AdvBench
#   harmless.txt  ← from Alpaca

# Sumandora ships no prompt sets by design.
# pick your own for your use case.

The three stages, in code

M1 has three stages: collect activations, extract a direction, apply it. Each stage is small - the whole procedure fits comfortably in fifty lines of Python. What follows is the actual code from published implementations, not a paraphrase.

Stage 1: Activation collection

Load the model. Take a sample of harmful prompts and a matched sample of harmless prompts. Tokenize each with the model's chat template. Run the model forward on each prompt, and record the residual stream activations at the last token position of each layer.

The residual stream is the running vector that every transformer layer reads from and writes to. Each transformer layer is a delta added to it. Sampling activations there gives you a picture of the model's internal state at the moment it is about to produce a response - about to refuse or about to comply, depending on the prompt.

Stage 2: Direction extraction

# compute_refusal_dir.py

def compute_refusal_dir(
    model, tokenizer, layer_idx,
    harmful, harmless,
):
    harmful_states  = []
    harmless_states = []

    for prompt in harmful:      # 32 by default
        with torch.no_grad():
            out = model(
                **tokenize(prompt),
                output_hidden_states=True,
            )
            harmful_states.append(
                out.hidden_states[layer_idx][0, -1, :]
            )

    for prompt in harmless:     # 32 by default
        with torch.no_grad():
            out = model(**tokenize(prompt),
                        output_hidden_states=True)
            harmless_states.append(
                out.hidden_states[layer_idx][0, -1, :]
            )

    diff = (torch.stack(harmful_states).mean(0)
          - torch.stack(harmless_states).mean(0))
    return diff / diff.norm()

Several free parameters live in this stage. Which layer's direction - Sumandora hard-codes 60% depth; Labonne's notebook extracts a candidate from every layer and asks you to eyeball which one works best (his run picked candidate layer 9); mid-to-late layers (0.4-0.9 of depth) tend to be where the refusal signal is cleanest. How many prompts - Sumandora uses 32 pairs, Labonne caps at 256, huihui-ai reports that on some models 32 pairs work better than a larger set. Which residual position - Arditi's original code measured three positions (pre, mid, post); FailSpy's measured two; Sumandora accesses one via hidden_states. For a first run, take the tool's default.

Stage 3: Ablation

Once you have the direction, there are two mathematically equivalent ways to apply it. Inference-time ablation installs a hook that subtracts the direction's component from every hidden state during each forward pass. Weight orthogonalization bakes the same operation into the stored weights themselves. Arditi and colleagues prove the equivalence; the choice is whether you want the modification to travel with the file.

# inference.py

def make_ablation_hook(refusal_dir):
    def hook(module, input, output):
        if isinstance(output, tuple):
            hidden = output[0]
        else:
            hidden = output

        # project onto refusal_dir, subtract
        proj = (hidden @ refusal_dir) \
                 .unsqueeze(-1) * refusal_dir
        hidden -= proj

        return output
    return hook

# install one hook on every layer:
for layer in model.model.layers:
    layer.register_forward_hook(
        make_ablation_hook(refusal_dir)
    )

To make the change permanent - to ship a model that others can download and run without needing your hooks - you orthogonalize the weights directly. The canonical helper is Labonne's get_orthogonalized_matrix, adapted from FailSpy's ortho_cookbook.ipynb:

# labonne / AutoAbliteration
# (adapted from FailSpy/abliterator)

def get_orthogonalized_matrix(W, refusal_dir):
    """Subtract the part of W that would
    write along refusal_dir. Returns the
    edited matrix; original is untouched."""
    proj = (W @ refusal_dir).unsqueeze(-1) \
             * refusal_dir
    return W - proj

# apply to every place that writes:

model.embed_tokens.weight.data = \
    get_orthogonalized_matrix(
        model.embed_tokens.weight,
        refusal_dir)

for block in model.model.layers:
    block.self_attn.o_proj.weight.data = \
        get_orthogonalized_matrix(
            block.self_attn.o_proj.weight,
            refusal_dir)

    block.mlp.down_proj.weight.data = \
        get_orthogonalized_matrix(
            block.mlp.down_proj.weight,
            refusal_dir)

# save; the file now travels
# without refusal.
model.save_pretrained("./abliterated/")

What it costs, what it preserves

The original paper's central empirical claim is that removal is surgical. Refusal, the authors show, is mediated by a one-dimensional subspace across 13 open-source chat models up to 72B parameters, spanning the Qwen, Yi, Gemma, Llama-2, and Llama-3 families from 1.8B upward. Weight orthogonalization pushes attack-success rates on harmful behaviors up to levels comparable with dedicated prompt-based jailbreaks, while general-capability benchmarks barely move. MMLU, ARC, and GSM8K typically change by under one point in either direction; the paper reports MMLU on Llama-3 70B moving from 79.9 to 79.8, a swing indistinguishable from noise.

The consistent exception is TruthfulQA. Across nearly every model tested, TruthfulQA drops after M1: -2.3 points on Llama-3 70B, -3.5 on Yi 34B, comparable magnitudes elsewhere. This is the empirical seed of the truthfulness-cost theme that recurs across all eight methods. The direction that mediates refusal is entangled with the model's disposition to push back on plausible falsehoods; cutting one takes off part of the other.

Different practitioners report different results for a mundane reason: the method has free parameters, and small choices change the outcome. This is why the automated variant (Heretic) exists - to search over the parameter space rather than have a human pick.

Canonical example models

failspy/llama-3-70B-Instruct-abliterated, updated May 2024, is the reference large-scale M1 artifact: Llama-3 70B Instruct with the refusal direction orthogonalized out and no repair step. Among the first abliterations to draw wide download traffic, and still a common base for downstream merges.

huihui-ai/Qwen3-8B-abliterated is representative of the highest-volume M1 producer. huihui-ai uses a fast difference-of-means variant that does not go through TransformerLens, ships GGUF and Ollama builds alongside the safetensors, and typically releases within hours of a base model's launch. The model card explicitly labels the work a "crude, proof-of-concept" refusal removal - unusually candid provenance.

huihui-ai/Huihui-Qwen3.8-27B-abliterated shows M1 shading into M3: only a subset of layers is ablated, with the first 15 layers left untouched. The producer files it as plain abliteration, but its selective-layer procedure is the M3 idea in practice.

Verify it actually worked

Two checks. First, count refusals on a held-out set. Run the model against the test split of harmful_behaviors and count how many prompts produce a refusal template versus a substantive answer. A good M1 on a small model pushes refusals from 90+/100 down to a few/100.

Second, confirm you did not damage capability. Run at least one capability benchmark - MMLU is standard - and compare against the base model. Anything above 1-2 points of drop is a warning sign; anything above 5 points is a failed run. See the benchmarks article for how to read the numbers.

huihui-ai's Ornith-1.5-9B-Instruct-abliterated card notes that ablating only the first 20 of 32 layers was sufficient there, using the same 32-sample dataset - a useful data point for calibrating what "enough" looks like at that model size.

When it does not work

Three failure modes recur.

Same responses as before (no effect). Reported repeatedly on Labonne's article for newer small models (Llama-3.2-3B, some Qwen variants). Cause is usually one of: wrong layer module path (Sumandora's README warns some Qwen builds expose layers as model.transformer.h rather than model.model.layers), a chat template mismatch (the tokenizer's default template does not match what the model was trained on), or a model where a single direction is genuinely insufficient. Fix: correct the module path; try a different layer; try a different chat template; move to a multi-direction method.

Single direction insufficient on larger models. Research from the RFM-AGOP paper (arXiv:2607.02396) shows Qwen3 8B needs at least three ablated directions to pass a 50% attack-success threshold. A single direction suppresses refusal in small models reliably but not in models of 8B and above. Fix: ablate the top-k directions rather than only the top one, or use a tool that does (Heretic's optimizer effectively covers a small band of related directions; M3 layer-wise selection can also help).

Coherence damage. Too aggressive an edit, or ablating too many or too-early layers, produces degraded prose - the model becomes rambling, off-topic, or grammatically loose. Fix: ablate fewer layers; try a projected or norm-preserving variant (Jim Lai's norm-preserving biprojected abliteration); if the damage persists, run a DPO healing pass on the output.

Cost and time for a real run

Concrete numbers, as of July 2026 rented-hardware pricing:

• 7-8B model on RTX 3090 or 4090 ($0.50-0.69/hr on Runpod): 10-20 minutes end-to-end including download, well under $0.50 total.
• 13B model on a 24 GB card: 20-40 minutes, roughly $0.50-1.00.
• 30B model on A100 80 GB ($1.39/hr on Runpod): 1-2 hours, $1.50-3.00.
• 70B model on A100 80 GB in int4 for extraction with shard-streamed orthogonalization: 1-3 hours, roughly $2-5.

Cost is dominated by GPU rental time; the direction-extraction computation itself is a small number of forward passes. If you are running M1 repeatedly (across a batch of models, or across parameter choices for a single model), Heretic's automated parameter search is likely a better use of the same hardware budget.

Related literature

The Q3 2026 academic corpus review verified 27 primary sources for the field of refusal removal. M1 is the central node - Arditi 2024 defines it, and nearly every other paper in the corpus either builds on it, refines it, defends against it, or measures it. This section organizes the M1-relevant literature by role. Full bibliographic entries with arXiv IDs are in the wiki references page.

Precursors and foundational lineage

Immediate antecedents to Arditi 2024. These papers established that steering a language model by adding or subtracting activation differences was a viable class of intervention on aligned models, and provided the difference-of-means construction that M1 adapts.

• Turner, A. M., Thiergart, L., Leech, G., et al. (2023). Activation Addition: Steering Language Models Without Optimization. arXiv:2308.10248. Introduces ActAdd: inference-time control via activation differences from paired prompts, preserving off-target performance. Cited by CAA and RepE below.
• Rimsky, N., Gabrieli, N., Schulz, J., et al. (2023). Steering Llama 2 via Contrastive Activation Addition. ACL 2024 (Long Papers). arXiv:2312.06681. CAA averages the difference in residual stream activations between pairs of positive and negative examples of a behavior. Direct precursor to the difference-of-means construction M1 uses. Nina Rimsky also publishes as Nina Panickssery, listed as an Arditi 2024 co-author.
• Zou, A., Phan, L., Chen, S., et al. (2023). Representation Engineering: A Top-Down Approach to AI Transparency. arXiv:2310.01405. Establishes the general framing that concept vectors are first-class objects one can read and control. Cited by Arditi 2024 as a direct antecedent.

Model-editing lineage

The rank-one weight-editing framing that Arditi 2024 uses to orthogonalize refusal out of the model traces to the ROME/MEMIT lineage from the Meng and Bau group. These are not activation-steering papers, but they established the mechanical vocabulary that made M1 possible.

• Meng, K., Bau, D., Andonian, A., Belinkov, Y. (2022). Locating and Editing Factual Associations in GPT (ROME). NeurIPS 2022. arXiv:2202.05262. Rank-One Model Editing on feed-forward weights for specific factual associations. Direct antecedent to the rank-one framing later adopted by M1.
• Meng, K., Sharma, A. S., Andonian, A., et al. (2022). Mass-Editing Memory in a Transformer (MEMIT). ICLR 2023. arXiv:2210.07229. Multi-layer editing extension that scales ROME to thousands of associations on GPT-J and GPT-NeoX.
• Elhage, N., Nanda, N., Olsson, C., et al. (2021). A Mathematical Framework for Transformer Circuits. Transformer Circuits Thread. transformer-circuits.pub. Grounds the residual-stream framing that M1 operates on. Cited by Arditi 2024. Not on arXiv - primary URL only.

Refinements and multi-directional critiques (M9 review track)

Five independent research groups have shown that the single-direction claim in Arditi 2024 is a first approximation. Refusal appears to occupy a cone or a subspace rather than a line, and different categories of refusal correspond to distinct directions that nevertheless act as a shared control knob. These papers are the primary sources for the M9 (multi-directional / subspace) review track opening for the next quarterly report.

• Wollschläger, T., Elstner, J., Geisler, S., et al. (2025). The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence. ICML 2025. arXiv:2502.17420. Shows there exist multi-dimensional polyhedral cones containing infinite refusal directions rather than a single one. Introduces "representational independence" and a gradient-based refinement.
• Piras, G., Mura, R., Brau, F., et al. (2025). SOM Directions are Better than One: Multi-Directional Refusal Suppression in Language Models. AAAI 2026. arXiv:2511.08379. Uses Self-Organizing Maps to extract multiple refusal directions; proves SOMs generalize the difference-in-means technique. Multi-direction ablation outperforms the single-direction baseline.
• Joad, F., Hawasly, M., Boughorbel, S., et al. (2026). There Is More to Refusal in Large Language Models than a Single Direction. arXiv:2602.02132. Eleven categories of refusal behavior correspond to geometrically distinct directions, yet all act as "a shared one-dimensional control knob."
• Winninger, T. (2026). Fast Multi-dimensional Refusal Subspaces via RFM-AGOP. Mechanistic Interpretability Workshop at ICML 2026. arXiv:2607.02396. Recursive Feature Machine with Average Gradient Outer Product identifies multi-dimensional refusal subspaces in seconds on the Qwen 3 family (1.7B, 4B, 8B, 14B) and Qwen 2.5. Single author, Saclay France.
• Wang, X., Wang, M., Liu, Y., Schütze, H., Plank, B. (2025). Refusal Direction is Universal Across Safety-Aligned Languages. arXiv:2505.17306. A refusal direction extracted from English bypasses refusals in 14 other languages with near-perfect effectiveness, without additional fine-tuning. Extends M1 cross-lingually.

Defenses, robustness studies, and restoration

Papers on defending an aligned model against M1, evaluating M1 robustness across pretraining conditions, or restoring refusal after abliteration. The two restoration methodologies (ROSI, AMRA) update earlier statements in the wiki that no published restoration methodology existed - see Case 3 in boundary-cases.

• Abu Shairah, H., Hammoud, H. A. K., Ghanem, B., Turkiyyah, G. (2025). An Embarrassingly Simple Defense Against LLM Abliteration Attacks. arXiv:2505.19056. Extended-refusal dataset distributes the refusal signal across multiple token positions so no single direction dominates. Reports refusal rate drops of at most 10% versus 70-80% in baseline models under M1 attack. KAUST/AUB.
• Agnihotri, S., Jakubassa, J., Dey, P., et al. (2025). A Granular Study of Safety Pretraining under Model Abliteration. NeurIPS 2025 Workshop Lock-LLM. arXiv:2510.02768. Empirical evaluation of M1 robustness across a granular sequence of safety pretraining checkpoints on SmolLM2-1.7B.
• Abu Shairah, H., Hammoud, H. A. K., Turkiyyah, G., Ghanem, B. (2025). Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection (ROSI). arXiv:2508.20766. White-box rank-one edit steering activations toward the refusal-mediating subspace - explicitly "the opposite" of M1. Increases refusal rate on Llama Guard 3 while preserving MMLU/HellaSwag/ARC.
• Truong, N. (2026). Abliteration Mitigation via Refusal Aliases (AMRA). arXiv:2608.18093. Rank-k updates to residual stream writer matrices; replaces refusal-inducing activations with random aliases. +2.16 pp post-abliteration refusal on Llama-3-8B, +14.70 pp on Gemma-2-9B, under 0.5 pp MMLU degradation. Single author.

Comparative studies and boundary papers

Empirical comparisons across M1-family tools, and papers that sit adjacent to M1 - close enough to be routinely confused with it in casual discussion but distinct in mechanism.

• Young, R. J. (2025). Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation. arXiv:2512.13655. Evaluates four tools (Heretic, DECCP, ErisForge, FailSpy) across sixteen instruction-tuned models (7B-14B). Source of the frequently cited Heretic GSM8K measurement (-7.81 pp average) and the covert-non-compliance observation. Single author, UNLV Neuroscience.
• Zou, A., Wang, Z., Carlini, N., et al. (2023). Universal and Transferable Adversarial Attacks on Aligned Language Models (GCG). arXiv:2307.15043. Adversarial suffix attack that transfers across models. Arditi 2024 mechanistically analyzes how GCG-style suffixes suppress the refusal direction. Adjacent to but distinct from M1 - see the wiki article on jailbreak vs abliteration.
• Lermen, S., Rogers-Smith, C., Ladish, J. (2023). LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B. arXiv:2310.20624. Refusal removal by fine-tuning rather than activation ablation - refusal rates about 1% on 70B for under $200. Contrast case for M1 - see fine-tuning vs abliteration.
• Bartoszcze, L., Munshi, S., Sukidi, B., et al. (2025). Representation Engineering for Large-Language Models: Survey and Research Challenges. arXiv:2502.17601. Broad survey of representation engineering; treats abliteration as one application within the wider family. Does not adopt the M1-M8 classifier used in this wiki.

Where to go next

If your M1 run leaves residual refusals, move to M3 (layer-wise selection) or Heretic (automated multi-parameter search). If it damaged capability, run a DPO healing pass (M4). If you want a file that runs on a laptop, quantize the output to GGUF (M8). If you want to combine your abliterated model with others, use mergekit (M5).

For end-to-end workflow that stitches these together with concrete commands and cost math, see the practical how-to article.