Known gaps in the abliteration wiki
A running inventory of eight techniques that appear in model cards but have no dedicated wiki article, no released implementation, or no consolidated documentation. Each entry names what is known, where it surfaces, why it has not been written up, and what would need to happen to close the gap. This page is both an honest inventory and an invitation.
Eight techniques the abliteration community uses without having consolidated documentation: SSM conv1d repair for hybrid Mamba, MTP head graft on DeepSeek, abliterix and other named-once experimental tools, FP8 bake-iteration for abliteration under aggressive quantization, non-constant per-layer weights on Gemma, grimjim norm-preserving biprojected abliteration, CHES preprocessing for DPO healing sets, and automated multi-directional ablation for models above 8B. Each has a card below describing what is known and what closing the gap would look like. If you can close one, the wiki is open.
- What each of the eight gap-tracking entries covers
- Why some techniques get documented and others do not
- What contributing a wiki article to close a gap actually requires
- How the field grows through documented craft rather than only papers
The wiki cluster on this site covers what the field has consolidated. It does not cover everything the field does. Some techniques are used routinely in model cards, mentioned in Discord discussion, and cited in adjacent papers, but have never been written up as a dedicated article, packaged as a released tool, or consolidated as documentation a practitioner can read and act on. This page tracks those gaps.
Each entry below names what is known, where the technique surfaces, why it has not been documented so far, and what would need to happen to close the gap. Reading the list gives you a sense of where the wiki is honestly weak; contributing an article that closes one is welcome.
The eight gaps
SSM conv1d repair for hybrid Mamba architectures
On hybrid Mamba architectures (Jamba, Zamba, some Falcon variants) the standard orthogonal-projection ablation can leave the SSM path producing degraded output while the transformer path recovers. Practitioners have reported that reinitialising or partially restoring the conv1d weights in affected layers fixes the SSM outputs. Some huihui-ai releases mention "conv1d selectively preserved" without further detail.
huihui-ai hybrid-model cards, occasional Discord discussion in Chinese practitioner channels, adjacent to the Heretic README note that SSM and hybrid Mamba are unsupported.
Hybrid architectures are a small share of the released model corpus (under 5 percent). The practitioners running them have solved the problem in-house and moved on. No one has taken the time to consolidate the fix into a repeatable recipe.
A practitioner with access to a hybrid model (Jamba-mini would work) and enough VRAM to run the ablation, healing, and comparison. Would produce: a diff of which conv1d parameters to preserve, before-and-after outputs on 20-30 probe prompts, and a KL comparison versus base. Publish as a wiki article and link from best-practices architecture rule 3.
MTP (multi-token-prediction) head graft on abliterated bases
DeepSeek-V3 and DeepSeek-R1 use multi-token-prediction heads to accelerate inference through speculative decoding. Standard abliteration touches the output tensor and the primary head; the MTP head is often overlooked and can either be silently damaged (bad speculations, hidden by fallback to the primary head) or left unchanged (in which case its speculations occasionally trigger refusals the primary path no longer expresses).
DeepSeek model cards briefly. Community discussion around huihui-ai DeepSeek-R1 abliterations. Not in the Heretic or Sumandora codebases.
The MTP mechanism is DeepSeek-specific and requires understanding both abliteration and DeepSeek architecture. Most abliteration practitioners work with Llama or Qwen where the issue does not arise; most DeepSeek practitioners are not doing abliteration.
A DeepSeek-V3 or R1 abliteration where the MTP head is inspected before and after. Document: does the MTP head need to be ablated with the same direction as the primary head, left untouched, or removed. Measurement matters: run the model with and without speculative decoding on refusal probes.
abliterix and other experimental refusal-editing tools
abliterix is one of several tools mentioned in model cards from mid-2025 as an alternative to the Sumandora and FailSpy pipelines. Beyond the name and the reference, nothing is publicly documented about what it does differently. Similar patterns exist for at least three other named tools that appear once and are not heard from again.
One or two model cards each. No repo, no readme, no comparison against Heretic or Sumandora.
These tools are either private forks the author did not release, abandoned experiments the author considered failed, or repackagings of Sumandora with cosmetic changes. Without the source it is impossible to know which.
Contact from anyone who used or built abliterix or the other named-once tools with even a brief description of what they did differently. Alternatively, the reader of a card that references such a tool who has the time to reach out to the model author and ask.
FP8 bake-iteration for abliteration under aggressive quantization
FP8 (E4M3 or E5M2) quantization of an abliterated model can partially reactivate refusals that were successfully ablated at BF16. The quantization error on ablation-target tensors is enough to move some directions back into the refusal-signalling subspace. Some practitioners report running a second-pass ablation on the FP8 checkpoint to reinforce the direction; others report that the second pass damages coherence more than it removes reintroduced refusal.
Anecdotal in Discord and Reddit. Occasionally referenced in FP8-release model cards without detail.
FP8 is still emerging as a distribution format. Most abliteration releases stop at BF16 or use GGUF Q8_0, which does not exhibit the same error pattern. The subset of practitioners who ship FP8 abliterations is small.
Systematic before-and-after: ablation at BF16, quantize to FP8, measure refusal rate, run second-pass ablation on the FP8 model, measure again. Compare with re-ablation at BF16 followed by FP8 quantization (different order). Report on a 3-5 model sample so the results are not coincidental.
Non-constant per-layer ablation weights on Gemma-3 and Gemma-4
mlabonne released gemma-3-12b-it-abliterated-v2 with non-constant per-layer ablation weights and noted the choice was empirical (uniform weights produced worse coherence). The specific schedule (which layers get which multipliers) is not published in the model card and not visible from the released weights alone.
mlabonne blog post about the v2 release, without the schedule. Heretic technically explores per-layer weights during its optimisation but does not emit the winning schedule as documentation.
The schedule is model-specific and, on Gemma, sensitive to base-model checkpoint version. What worked on gemma-3-12b-it may not work on gemma-4-9b. Publishing the schedule risks readers copying it and getting worse results on their target model.
A convention for how to document a per-layer weight schedule: as a table? A visual? An included YAML? Plus a discussion of what the schedule captures that a scalar weight does not. Would extend best-practices architecture rule 4 (Gemma quirks).
grimjim norm-preserving biprojected abliteration
grimjim published this refinement in mid-2025 and it appears in the community best-practices list (ablation rule 5). The technique is documented at method level (edit o_proj and down_proj rank-2L simultaneously, preserve norms) but the practical implementation - how to compute the joint ablation direction, how to choose the rank, how to validate the preserved norms - is scattered across a Hugging Face blog post and Reddit comments.
grimjim Hugging Face blog. r/LocalLLaMA thread. Not consolidated as a wiki article.
grimjim treats the technique as one of several refinements they are exploring; consolidating one refinement into a canonical article may misrepresent their broader research direction. The wiki cluster would benefit from doing it anyway, credited clearly to grimjim.
A short wiki article covering: what norm-preservation means at the tensor level, why biprojected editing outperforms sequential editing of the two projections, the rank-2L rank choice, and a diff-versus-Arditi comparison. Would sit at /wiki/grimjim-biprojected-full/ alongside the existing /wiki/grimjim-refinements/.
CHES preprocessing for DPO healing sets
The Razin et al. paper on likelihood displacement (arxiv:2410.08847) shows that DPO on preference pairs with high CHES scores can cause preferred-response probability mass to move off the intended target rather than toward it. Filtering to the low-CHES 5-15 percent of pairs avoids this. The recipe is documented in the /datasets/heal recipes but the CHES score computation itself is a graduate-level formula, and no reference implementation of the filter has been released as a maintained tool.
The Razin paper. The abliteration.org heal recipe references the paper but does not link to a working implementation.
The filtering is done in-house by practitioners who have implemented the CHES calculation once and moved on. No one has published the reusable helper.
A small released tool (single Python file or notebook) that takes a preference dataset and a target model and returns the low-CHES subset. Would let every abliteration release apply the filter without reimplementing it. Ten to twenty lines of core code plus the arithmetic.
Automated multi-directional ablation for models above 8B
The Fast Multi-dimensional Refusal Subspaces paper (arxiv:2607.02396) shows Qwen3-8B needs at least three ablated directions to surpass a 50 percent ASR threshold using the RFM-AGOP method. Heretic supports multi-directional exploration in principle but its default TPE search is over single directions. Practitioners currently either accept the single-direction limit or run Heretic with multi-direction search manually and Pareto-select across the joint front.
The paper. Community best-practices ablation rule 4. Heretic advanced usage docs.
The default-versus-advanced gap in Heretic is fixable but has not been prioritised. The community best-practice says "expect models 8B and above to need multiple directions" without giving the reader an automated tool that just does it.
Either a Heretic default change (multi-direction search on models above 8B) or a wrapper tool that runs Heretic in the correct mode and consolidates the output. Plus wiki documentation of when the extra directions actually matter versus when single-direction suffices.
Why keep this inventory
Two reasons. First, honesty. The wiki cluster reads as if it covers the field completely; it does not, and pretending otherwise would misrepresent what the reader has. Naming the gaps explicitly lets readers calibrate. When a model card references a technique not covered here, the reader can check this page and see whether the gap is known.
Second, invitation. A wiki grows through documented craft, not only through papers. Every entry below is a place where a working practitioner with two or three hours could produce a document that becomes the reference for that technique. The barrier is not access to weights or compute - most of these gaps could be closed by consolidating what is already known into a single readable page.
How to contribute a gap-closing article
The wiki accepts articles that close listed gaps. The article should follow the existing wiki-cluster format: a title with an accent word, a deck describing the technique in two sentences, a quick answer, a learn list, a body organised as h2 sections with anchor ids, and either references to primary sources or (for craft-knowledge articles) named practitioners whose model cards or discussion threads document the technique. Length varies; 6-10 minutes reading time is typical.
Once the article exists, this gap-tracking page will be updated to move the entry from "known gap" to "closed, see /wiki/{new-slug}/". Historical entries are kept rather than deleted, so the closure record is visible.
See also
Community best practices for the 23 rules that are consolidated. Non-English communities for the parallel case of language-community knowledge that the English-language wiki systematically missed. Tools of the field for the released tools that gap-tracking entries most often reference.