A running list of 25 conventions the abliteration community has converged on since the technique became tractable. Each entry is a rule with a one-paragraph reason and a source, grouped by workflow stage. This is not a tutorial; it is what practitioners cross-check against before publishing a release.
The 25 rules below sit above the recipes: they apply across techniques and stages. The most consequential of them are: optimise for KL divergence and not just refusal count; leave layer 0 unablated; use difference-of-means rather than PCA; expect models 8B and above to need multiple directions; preserve layer norms during ablation (MPOA); heal with a light DPO/ORPO pass and expect GSM8K to lag; never train exclusively on harmful examples; treat old benchmarks as historical rather than absolute; and remember that DPO bypasses vectors rather than removing them, so an uncensored model is not necessarily a safe one.
In this article
Extraction: 5 rules
Ablation: 5 rules
Healing: 4 rules
Evaluation: 4 rules
Data hygiene: 2 rules
Architecture-specific: 4 rules
After release: 1 rules
The recipes on the category pages tell you how to do a specific thing. The list below tells you what to remember while doing it, across techniques. Each entry started as a hard-won observation by one or more practitioners, was reproduced enough times to become convention, and now shows up as a default in the current-generation tools (Heretic, grimjim's llm-abliteration, Sumandora's implementation) or as a warning in the reflective literature.
Scan the section for whichever stage you are working in. The rules are ordered roughly by how consequential the community treats each one; near the top of every section is the rule you are most likely to regret not knowing.
Extraction
What to do when building the pair and computing the refusal direction.
Use difference-of-means, not PCA, for the direction
The standard implementations (Sumandora, FailSpy, Heretic) compute harmful_mean minus harmless_mean at the final token. Simpler, empirically more robust than PCA on the same activations, and the field has converged on it for a reason.
A few hundred prompts per side is enough. Thousands are worse
The direction estimator saturates fast. Sumandora, FailSpy, and Heretic all use hundreds, not thousands; Cracken reported clean results with 52 harmful / 73 harmless on a domain sweep. Bigger sets averaged in more noise from framing and length variation.
If harmful is imperatives ("Write instructions for X") and harmless is questions ("What is Y"), the direction encodes imperative-vs-interrogative as much as refuse-vs-comply. Ablating suppresses ordinary commands, not just refusals. Match imperative-to-imperative, question-to-question.
Refusal responses tend to be short and compliant responses tend to be longer. If harmful prompts are shorter than harmless prompts, the extractor bakes some length signal into the direction and ablation shifts output length across the board. Keep mean and std within 10 percent.
Use the positive/negative token trick to steer the direction
FailSpy: specify positive_toks like " Sure" and negative_toks like " cannot" so the extractor optimises for the actual token transitions you want to see. Sharpens the target and is why the same 512 samples work well in FailSpy where they underperform in a bare difference-of-means.
Optimise for KL divergence, not just refusal count
Refusal count tells you the model stopped refusing. KL divergence tells you the model is still intelligent. Heretic co-minimises both and hits KL 0.16 on Gemma-3-12B where hand-made ablations of the same model land at 0.45-1.04. Same refusal suppression, one third to one sixth of the capability damage.
Ablate MLP down_proj more conservatively than attention o_proj
Weidmann: "MLP interventions tend to be more damaging to the model than attention interventions." Heretic separates ablation parameters for the two components explicitly. If you have to sacrifice one, sacrifice attention edits before MLP edits.
Leave the earliest layers (and often layer 0) unablated
Early layers carry low-level linguistic representations that abliteration can garble; late layers carry task-specific integration. Middle layers carry refusal cleanly. huihui-ai keeps the first 15 layers untouched on 62-layer Qwen3.8-27B and specifically re-issued a Gemma build after "Changing the 0 layer to eliminate the problem of garbled codes."
For models 8B and above, expect multiple directions or multiple layers
Single-direction ablation is insufficient. Qwen3-8B needs at least three ablated directions to surpass a 50 percent ASR threshold (RFM-AGOP). grimjim rank-2L intervention edits both o_proj and down_proj across multiple layers simultaneously to overcome model self-repair.
Subtracting the whole direction changes weight magnitudes and damages reasoning. Removing only the directional component (Magnitude-Preserving Orthogonal Ablation, grimjim) preserves capability with the same refusal suppression. Now baked into Heretic as the default.
Heal with a light DPO/ORPO pass. Do not lobotomise
Post-ablation healing on mlabonne/orpo-dpo-mix-40k recovers most of the capability drop with orpo_alpha 0.1 and one epoch. Over-training the DPO stage reinstalls refusal patterns or damages instruction-following. Light is right.
Expect GSM8K to lag after healing. The reference mix is math-poor
orpo-dpo-mix-40k contains only about 2,200 math-preference samples out of 44,245 total. Every abliterated release documents that GSM8K under-recovers relative to MMLU, ARC, and HellaSwag. Add math preferences to the mix if math capability matters for your release.
Filter DPO pairs by CHES score to avoid unintentional unalignment
Pairs where chosen and rejected are the same response type (two refusals, or two compliances) cause likelihood displacement: probability mass shifts off the preferred response during DPO. Keeping the bottom 5 percent by length-normalised CHES score gets rid of these pairs and prevents the pathology.
DPO bypasses toxic vectors rather than removing them
Healing hides but does not delete. A DPO-healed abliterated model has the ablated behaviours dormant in weight space, and they can be reactivated by a small counter-training run. This is a feature for reversibility and a risk for permanence claims. Do not claim "safe" from a DPO healing pass alone.
Either number alone lies. 3/100 refusals sounds like a win until KL is 1.04. KL 0.16 sounds clean until you notice the model still refuses 40 out of 100. Practitioners now expect both numbers in release notes as a matter of course.
HarmBench explicitly requires that attack and defense methods not be fine-tuned on the test split. Doing so leaks benchmark behaviours and inflates apparent success. Keep XSTest strictly for over-refusal measurement, never as training data.
Judge panels disagree. Use two judges and report the range
HarmBench-Llama-2-13B-cls, JailbreakBench-judge, and StrongREJECT can bracket the same completion with 30 percent spread in ASR. Newer models emit an affirmative prefix then self-correct, and single-judge scores can miss this. Report ASR-H, ASR-J, ASR-S separately when accuracy matters.
AdvBench in 2026 does not measure what AdvBench in 2023 measured. Base models learned different refusal patterns; the same 520 prompts elicit scattered responses. Treat 2023-era benchmark numbers as historical baselines, not absolute measurements.
What to do (and not do) with the corpora themselves.
Never train exclusively on harmful examples
Training only on the harmful side of an alignment corpus imprints apology templates and toxicity. WildGuardMix explicitly recommends against this: "It is recommended not to train a LLM exclusively on the harmful examples." Keep the benign-that-looks-harmful contrast in every training mix.
Set aside 20 percent of both harmful and harmless sides. Do not touch during extraction. Evaluate after ablation. Without a held-out split, "my abliteration worked" is a claim about the model on the same prompts you used to define what its behaviour should be, which is a tautology, not evidence.
What differs when your model is not a vanilla dense transformer.
On MoE models, target routed-expert down_proj matrices
The refusal direction concentrates in the routed-expert down_proj rather than in dense layers on Mixture-of-Experts architectures. Single-layer dense recipes fail on GLM-5.3-Flash and similar. Intervention must span the routed experts across many layers.
On hybrid/SSM models, plan for MTP head loss and conv1d repair
Abliteration tooling can silently drop multi-token-prediction heads on hybrid Mamba/attention models (Qwen3.5-style), and SSM conv1d outliers may need a repair pass. Heretic does not support SSM/hybrid Mamba at all. AEON-7 documents the manual graft and outlier repair.
After abliterating an FP8 model, requantize byte-for-byte
FP8 abliteration accumulates leakage across matrices. OrcaRouter documents 0.188 harmful refusal at 8 bake iterations improving to 0.094 at 32 iterations across 12,442 FP8 matrices. After the abliteration, requantize byte-for-byte to the original FP8 scheme so it serves on the identical vLLM kernel path.
Fine-tuning an abliterated model on safety data reintroduces refusals
The ablated behaviours persist dormant in weight space. Any subsequent fine-tune on refusal-heavy or safety data partially reactivates them. If you want the model to stay uncensored, use neutral task data for downstream fine-tunes.
No related articles yet. As new pieces are added to this cluster, they'll appear here.
Catalog is the map. Apps are the tools.
Run models on your own machine, not in the cloud.
Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.