grimjim's refinements

The context

Arditi's original orthogonalization is a straightforward operation. For every weight matrix W that writes into the residual stream (attention output projections, MLP output projections, embedding), compute W' = W - r̂ (r̂ᵀ W), where r̂ is the unit refusal direction. The matrix loses its ability to produce any component along r̂. The refusal capacity of the model, as far as the direction extraction found it, is gone.

The operation is mathematically clean, but it is also mathematically heavy-handed. The projection removes the r̂ component from every column of every affected weight matrix, whether or not that column was contributing to refusal. Columns that were doing other things but happened to have some r̂ component get their behavior modified by the same amount. In practice this shows up as a measurable but small capability drop on some downstream tasks - the "TruthfulQA drift" that Arditi's own paper documented, and a broader pattern of collateral degradation on capability benchmarks that has become a subject of practitioner attention.

Between October and November 2025, Jim Lai (grimjim on Hugging Face) published two blog posts and accompanying model releases that propose mathematical refinements to the ablation operation. Both refinements were absorbed into Heretic in early 2026. Because Heretic is now the largest producer of abliterated models by volume in our catalog, these refinements are the practical baseline for most abliteration produced in 2026.

Projected abliteration (25 October 2025)

Projected abliteration is the first refinement. The operation modifies the standard Arditi orthogonalization by projecting the intervention onto a smaller subspace: instead of removing the full r̂ contribution from every weight matrix column, remove only the component that is relevant to producing refusal outputs.

The mathematical form: for each weight matrix W, compute a projection P onto the subspace of W's column-space that has nonzero correlation with the refusal signal on the extraction set. Then apply the orthogonalization only within that subspace: W' = W - P (r̂ (r̂ᵀ W)). Columns that had no involvement with refusal on the extraction set are left untouched; only the columns that were actually contributing to the refusal pathway are modified.

Grimjim's empirical claim in the blog post: on the models tested, projected abliteration reduces the KL divergence between pre- and post-abliteration output distributions on harmless prompts by a factor of roughly two to three, compared to standard Arditi orthogonalization applied to the same model. Refusal removal is equally effective; collateral damage is smaller.

Norm-preserving biprojected abliteration (6 November 2025)

Norm-preserving biprojected abliteration adds a second correction to projected abliteration. The observation motivating it: even projected abliteration, by removing components from a weight matrix, reduces the matrix's Frobenius norm - which in transformer architectures is a stand-in for how much signal that matrix contributes to the residual stream. A smaller-norm matrix contributes less; capability suffers even when refusal is cleanly removed.

The correction: after the projected orthogonalization, rescale the modified matrix to restore its pre-edit Frobenius norm. Practically, this means applying a small uniform scaling factor to the entire matrix after the projection edit. The matrix loses its refusal component but retains the total "signal amount" it was contributing before.

Grimjim's evaluation on the norm-preserving variant reports further capability preservation compared to the plain projected version, at similar cost in refusal removal. Together, the two refinements produce an ablation that is closer to a "surgical" edit than the plain Arditi baseline.

How Heretic integrates them

Heretic v1.0.0 (November 2025) used the standard Arditi orthogonalization by default. Heretic v1.1 and later (through v1.2.0 in February 2026 with the LoRA-adapter mode) integrated projected abliteration and norm-preserving variants as selectable options in the pipeline. Recent Heretic releases run projected abliteration by default; norm-preserving is available as an optional flag. For a model produced by Heretic with default settings in 2026, the practical extraction is Arditi difference-of-means followed by grimjim's projected orthogonalization.

The consequence for our catalog: every Heretic-produced model is tagged as M1 direct removal (correct at the M1-M8 classification level), and the extraction methods classifier tags it as using difference-of-means (correct at the extraction level). The refinement - projected versus plain - is not currently a separate classifier axis, though it could become one in a future quarter if the distinction becomes load-bearing for readers.

The Young comparative study

In December 2025, R. J. Young (arXiv:2512.13655) published Comparative Analysis of LLM Abliteration Methods, a solo-author evaluation of the three ablation shapes across 16 models. The finding: no single geometric variant dominates. Standard orthogonalization, projected abliteration, and norm-preserving biprojected each perform best on some model families and worst on others. The outcome is strongly model-dependent, and a producer choosing among the three should test rather than assume.

Young's paper is the reference we cite when readers ask "which grimjim variant should I use?" The answer, in the current state of measurement, is that the choice matters and cannot be decided from theory alone; it needs empirical calibration per target model. Heretic's default of projected abliteration is a reasonable middle option, but not universally the best.

What these refinements are not

grimjim's refinements are refinements to the ablation step, not to the extraction step. They start from the same difference-of-means direction Arditi's method produces (or a variant thereof) and modify how that direction is removed from the weights. They do not address the single-direction versus cone-of-directions dimensionality debate; if refusal is mediated by multiple directions, both projected and norm-preserving abliteration remove only the one direction they were given, and leave the others latent, just like the plain Arditi method does.

For a Wollschläger-scale defense against multi-directional refusal, projected abliteration would need to be applied to each extracted direction separately (or the projected operation would need to be generalized to a projected subspace intervention). This has not been published as of Q3 2026; it is a natural extension the community might undertake.