Restoration and defenserestoration

The symmetry of the operation

Every intervention that removes refusal from a language model has a mathematical mirror that puts it back. This is not a philosophical observation - it is a straightforward consequence of the geometry described in the refusal direction article. If subtracting a rank-one edit from the weights makes a model stop refusing, then adding the same rank-one edit back (or applying it to a different reference model) makes a model start refusing more strongly. The direction that carries "no thanks, I cannot help with that" is the same direction, up to sign, that carries "here is why I must decline."

As of mid-2025, no formal restoration methodology existed in the published literature. That gap has since been closed. Between May 2025 and October 2025, four papers appeared that treat the symmetry as engineering rather than metaphor. This article documents them. None has yet been observed in a production pipeline in the catalog, but the wiki records them because the mathematical operation and the deployment operation are the same operation viewed from opposite directions, and both are load-bearing for the field.

ROSI: Rank-One Safety Injection (August 2025)

ROSI (arXiv:2508.20766, August 2025), formally titled Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection, is the first published method to restore refusal in a model that has already been abliterated. The mechanism is exactly what its title suggests: take a rank-one edit (the same shape as the Arditi orthogonalization) and apply it in the reverse direction, adding the refusal component back into the residual stream.

The paper's evaluation covers models that had been publicly abliterated (huihui-ai and mlabonne releases), demonstrating that ROSI can restore refusal behavior to near the base model's rate on the same evaluation set that abliteration reduced. The edit is portable: applying ROSI to one abliterated Llama model produces refusal behavior close to what applying the reference orthogonalization would restore, without needing to re-train or fine-tune. Rank-one, one-shot, weight-level.

What ROSI does not do: it does not defend a model against future abliteration. It restores refusal after the fact. A ROSI-treated abliterated model can be re-abliterated with the same procedure that produced the original abliterated version - the symmetry runs in both directions.

AMRA: Abliteration Mitigation via Refusal Aliases (June 2026)

AMRA (arXiv:2608.18093, June 2026), Abliteration Mitigation via Refusal Aliases, takes a different approach. Rather than restore refusal after removal, AMRA replaces the model's refusal signal with a decoy - an "alias" direction that abliteration procedures will find and remove, leaving the actual refusal behavior intact because it is now carried by a different underlying representation.

The mechanism, in loose terms: during a fine-tuning stage, the model is trained to emit refusal behavior that is triggered by activations along a chosen alias direction, while the natural refusal direction is progressively decorrelated from the refusal output. A difference-of-means extraction on this model will find the alias direction (because that is what statistically correlates with refusal outputs on the extraction set), and orthogonalizing against the alias will not remove refusal - it will only remove the decoy.

AMRA's evaluation reports that abliteration attacks on AMRA-treated models show near-zero attack success rate on the tested benchmarks, at a cost of a small capability regression that is smaller than the regression Extended Refusal introduces. The defense is model-specific: each protected model needs its own alias direction chosen during training.

Extended Refusal: Abu Shairah et al. (May 2025)

Abu Shairah et al. (arXiv:2505.19056, May 2025), An Embarrassingly Simple Defense Against LLM Abliteration Attacks, is the earliest of the four papers and the simplest. Its central observation: standard alignment training produces refusals that look statistically identical to each other - short declines with a small vocabulary of templates ("I cannot help with that", "As an AI assistant I must decline", and so on). Because these responses share a narrow representational signature, the difference-of-means direction picked out by an extraction pipeline is a clean, isolable signal.

Their defense: fine-tune the model to produce extended refusals - longer responses that include a brief acknowledgment of the topic, an explanation of the concern, and the refusal, in variable phrasing across categories. The refusal signal, distributed across a broader representational surface, is harder to extract as a single direction. Difference-of-means still finds something, but the extracted direction correlates less cleanly with refusal behavior, and orthogonalizing against it removes less of the model's refusal capacity.

The tradeoff is capability: Extended Refusal fine-tuning introduces a small drop on general benchmarks (they report about 1-3 points on MMLU across their tested models), which is the price of the broader representational surface. The method is compatible with any base model and requires no changes to inference infrastructure.

Agnihotri et al.: safety pretraining and Qwen3 (NeurIPS 2025 workshop)

Agnihotri and colleagues (NeurIPS 2025 workshop, arXiv:2510.02768), Granular Study of Safety Pretraining under Model Abliteration, investigate a different question: does safety-oriented pretraining (as opposed to instruction-time alignment) affect a model's abliteration-resistance?

Their finding, in the setup they tested: Qwen 3 practically does not lose capabilities after abliteration, in contrast to Llama 2 and Qwen 2.5 which lost measurable ground on capability benchmarks after the same procedure. The result is preliminary, based on a single model family per pretraining regime, and the causal claim (that safety pretraining specifically is what preserves capability) is not established. But the empirical observation is on record: at least one modern model, from a family known for safety-conscious pretraining, resists the capability-loss side of abliteration.

Whether this generalizes to a broader claim about safety pretraining as a robustness technique, or whether Qwen 3's result is specific to its architecture or training data, is an open question. The workshop paper flags it as an area for further study.

What is settled, what is open

Settled as of Q3 2026:

• Restoration is possible with a rank-one edit (ROSI). The symmetry of abliteration and restoration is not just theoretical; a working method exists.
• Defense against difference-of-means extraction is possible (AMRA, Extended Refusal). Both methods reduce abliteration attack success rate substantially in their evaluations.
• Not all base models are equally vulnerable to abliteration. Qwen 3 shows measurably different post-abliteration capability behavior than earlier Llama and Qwen models.

Open questions:

• Do these defenses survive multi-directional extraction (RFM-AGOP, SOM)? Both AMRA and Extended Refusal were evaluated against difference-of-means. Whether a multi-directional extractor would find alternate directions that survive the defense is not documented in the papers.
• Does ROSI restoration preserve the model's capabilities, or does the added rank-one edit introduce its own capability regression? The paper reports minimal degradation; independent verification is not yet published.
• Would production model producers adopt Extended Refusal or AMRA as a pre-release step, and if so, what would that mean for the abliteration pipeline? So far no released base model card documents having applied either defense.
• Is Qwen 3's abliteration-resistance a property of its safety pretraining, or of its scale, or of its instruction-tuning regime? The Agnihotri paper cannot separate these factors from a single model family.

Catalog implications

None of these methods currently appears in the M1-M8 classifier applied to models in our catalog. ROSI-restored models would need a new tag (something like "M1+ROSI" or an "abliteration reversed" flag) that we do not yet have. AMRA and Extended Refusal apply during base model training and would be tagged on the base model, not the derivative - and no base model in our catalog currently declares either defense in its model card.

The Wave 1 addition of the extraction_methods table tracks the geometric side of the operation. A parallel structure for restoration and defense operations may be added in a later quarter if adoption warrants it. As of Q3 2026 the ROSI, AMRA, Extended Refusal, and Agnihotri papers are documented here for reference but do not have DB structure of their own.