Datasets that shaped abliteration

Why five, not dozens

The abliteration.org catalog tracks 2,443 alignment datasets on Hugging Face. Most of them are variants, subsets, translations, or minor recombinations of a much smaller core. If you follow the citation graph and the model-card lineages backward, a striking pattern appears: nearly every abliterated model of the last two years used one or more of five specific corpora, either directly or through a downstream derivative that carried the original curatorial choices intact.

This is not an accident. Abliteration as a technique is unusual in that it depends on being able to show a model examples of what it should and should not refuse. A method that reads out an internal direction from activation differences needs a matched pair of prompts to differentiate. A method that heals capability loss needs a preference corpus. A method that verifies whether the ablation worked needs a benchmark. Each of these three needs was met early, canonically, and by a small set of authors whose choices then propagated everywhere.

What follows is not exhaustive. It is a short history of the datasets that shaped the field most, ordered by when they arrived and by what they made possible.

Anthropic hh-rlhf (2022)

Anthropic/hh-rlhf is the oldest dataset that still matters. Released alongside the paper Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback in April 2022, it contains roughly 170,000 human preference comparisons across two subsets: a helpfulness split and a harmlessness split. Each comparison presents two possible responses to the same prompt, with a human annotator marking one as preferred. The dataset was the training corpus for early Claude.

Its influence on abliteration is indirect but foundational. Every open-source instruct model that shipped with refusal behavior in 2023 and 2024 was trained on preference data that either was hh-rlhf, was derived from hh-rlhf, or was constructed in explicit imitation of its structure. When practitioners today speak of "the base model's refusal patterns," they are largely speaking of patterns learned from hh-rlhf-family data. Abliteration is the surgical reversal of a preference signal that hh-rlhf did more than any other single dataset to install.

The dataset also silently fixed the vocabulary of the field. The words helpful and harmless as opposing axes to be balanced, the framing of certain requests as inherently unsafe regardless of context, the notion that a refusal is a first-class output type worth optimizing for: these were curatorial choices, not natural categories, and they are baked into every model that trained on this data or its descendants. See what is refusal direction for how those choices manifest in the weights.

AdvBench (2023)

In July 2023 Zou, Wang, Kolter and Fredrikson published Universal and Transferable Adversarial Attacks on Aligned Language Models. Buried in the appendix was a small dataset called AdvBench: 520 harmful strings and 520 harmful behaviors, curated to represent categories the researchers considered worthy of refusal: violence, illicit substances, harassment, misinformation, self-harm.

The dataset itself is modest. What made it consequential was its role as the standardization event for the field. Before AdvBench there was no shared answer to the question "what counts as a jailbreak prompt?" Different papers used different prompt lists, results could not be compared, and the field lacked a common benchmark against which to measure attack success or defense robustness. AdvBench was small enough to be tractable, categorized enough to be interpretable, and public enough to be reused. Within a year it was the reference set that every subsequent jailbreak paper cited, and by extension the reference set that every abliteration paper had to demonstrate against.

Its 520 prompts became the workhorse of the extraction pipeline. Practitioners used AdvBench as the "harmful" side of the paired input set from which the refusal direction is computed, matching it against a corpus of ordinary instructions on the "harmless" side. The pairing was arbitrary in its details but stable in its form, and it defined what "computing the refusal direction" concretely meant for years afterward. See the Arditi method for the technical treatment.

AdvBench also shaped what abliteration could and could not detect. Because the 520 prompts covered specific topical categories, the direction extracted from them was a direction sensitive to those categories. Refusals of things outside AdvBench's coverage (novel harm framings, medical edge cases, satire, self-referential prompts) did not always sit on the same direction and were not always removed by the same ablation. This is not a defect of the dataset; it is what happens when a small, categorical corpus becomes the field's definition of a much larger phenomenon.

mlabonne's harmful_behaviors + harmless_alpaca (2024)

The most-used extraction split in the field today is a pairing rather than a single dataset: mlabonne/harmful_behaviors (a derivative of AdvBench's behavior list) and mlabonne/harmless_alpaca (a filtered subset of Stanford Alpaca instructions selected to be innocuous, task-focused, and phrasing-matched). Maxime Labonne published both in mid-2024 as companion inputs to his canonical abliteration tutorial.

The pairing solved a practical problem. Arditi's original paper had used AdvBench's harmful behaviors matched against Alpaca instructions, but there was no clean, uniform, ready-to-use version of that pairing. Practitioners assembled their own splits inconsistently, and results varied for reasons that had nothing to do with the ablation technique and everything to do with input curation. mlabonne's release standardized the split. By late 2024, "the standard extraction pair" and "mlabonne's harmful_behaviors + harmless_alpaca" were synonyms.

This has consequences that are rarely acknowledged. The refusal direction one extracts is not a property of the model alone; it is a property of the model relative to the specific paired inputs used to extract it. A model abliterated using mlabonne's pair is optimally purged of refusals to prompts that resemble mlabonne's pair. It may still refuse, unpredictably, categories that his harmful_behaviors did not cover. The field has largely settled on this pair as if it were canonical, and the resulting released models inherit its blind spots as a design feature.

orpo-dpo-mix-40k (2024)

The healing side of the pipeline has its own canonical corpus: mlabonne/orpo-dpo-mix-40k, a curated 40,000-pair preference mixture published in mid-2024. It combines pairs from several upstream sources (argilla's distilabel outputs, orca-style comparisons, capybara pairs) into a single split of moderate size, decent diversity, and known composition. It was the training corpus for Labonne's own NeuralDaredevil-8B, the release that demonstrated the two-stage abliterate-then-heal workflow.

Its adoption pattern mirrors AdvBench's. Once the two-stage recipe caught on, practitioners needed a healing corpus and orpo-dpo-mix-40k was the one Labonne had used, so it became the one everyone else used too. By early 2025 the pattern was established: if you were healing an M4-style release, you reached for orpo-dpo-mix-40k first and only substituted something else if you had a specific reason. See M4 for the workflow context.

The corpus's most consequential property, from an abliteration standpoint, is what it does not contain. Because it was curated for general instruction-tuning purposes rather than for uncensoring specifically, it does not deliberately weight compliance-over-refusal pairs. If a healing dataset contains many polite-decline responses marked as "chosen," a healing pass will partially reinstall refusal, the self-cancelling risk detailed in the DPO healing article. Practitioners have variously reported that orpo-dpo-mix-40k avoids this well enough in practice, and that it does not entirely avoid it. Both statements can be true simultaneously because the pipeline is a balancing act, not a clean fix.

WildGuardMix (2024)

The most recent entrant in the canonical five is allenai/wildguardmix, released by Allen AI in late 2024 as part of the WildGuard safety-classifier line. It contains roughly 92,000 prompt-response pairs annotated across three axes: whether the prompt was harmful, whether the response was harmful, and whether the response constituted a refusal. Unlike the earlier corpora, WildGuardMix explicitly separates the refusal-artifact question from the harmfulness question, allowing evaluations to distinguish "the model complied with a benign prompt" from "the model complied with a harmful prompt" from "the model correctly refused" from "the model over-refused a benign prompt."

This separation matters. Earlier evaluation pipelines conflated all four categories, treating any refusal as evidence of safety and any compliance as evidence of unsafety. That framing systematically penalized abliterated models for correctly complying with benign prompts that they had previously (over-)refused, and made over-refusal invisible as a failure mode. WildGuardMix's three-axis annotation, and the WildGuard classifier trained on it, made artifact-aware evaluation possible for the first time at scale.

The dataset also shifted what the field considered a good abliteration to look like. The old measurement was "attack success rate on AdvBench"; higher was better, up to a ceiling. The new measurement includes "over-refusal rate on XSTest-style benign prompts"; lower is better, and abliterated models typically outperform base instruct models here rather than being penalized. See reading benchmarks for how this reframing plays out in released evaluation numbers.

The datasets that shaped by absence

A history of the datasets that shaped the field would be incomplete without noticing which ones did not exist and how their absence shaped what got built.

There is no canonical multilingual extraction pair. mlabonne's harmful_behaviors is English. AdvBench is English. Most healing corpora are English or English-heavy. This is a large part of why abliteration outside English is still a smaller, more experimental practice; the tools work but the reference inputs do not exist in the same standardized form. See non-English communities.

There is no canonical domain-specialized extraction pair. If you want to abliterate a medical model without also abliterating its clinical caution, or a legal model without also abliterating its jurisdictional hedging, you must build your own pair. Very few practitioners do; most simply apply the general-purpose pair and accept whatever domain-specific side effects follow. The absence of domain-specialized pairs is why so many abliterated medical and legal models perform worse than their base versions on domain benchmarks.

There is no canonical compliance-first healing corpus. Everyone reaches for orpo-dpo-mix-40k because it is what exists and works. A corpus deliberately curated to weight capability recovery without reinstalling any refusal patterns would be a valuable contribution, and it has not been made at scale. See the heal category for what is currently available.

These absences are not neutral. They shape what abliterated models are, what languages they cover, what domains they serve well, and what failure modes they inherit. A dataset landscape different in these particulars would produce a fleet of open-weight models different in observable ways.

What this history tells us

The five canonical corpora share three features. Each was published with a clear technical purpose. Each was small enough to use without a compute budget. Each arrived early enough to become the default answer to a question the field was just beginning to ask. None of this is unusual for how technical communities coalesce around shared infrastructure; it is unusual only in how visible the pattern is when the infrastructure in question is training data rather than code libraries.

The consequence is that the shape of abliteration as a practice is largely the shape of these five datasets projected through several years of derivative work. Choices made by Anthropic annotators in 2022, by Zou's group in 2023, by mlabonne in 2024, and by Allen AI in late 2024 propagate forward into every abliterated release of 2025 and 2026. When we look at the /datasets catalog and see 2,443 corpora across six workflow stages, we are looking at a large landscape whose canonical five references still do most of the load-bearing work.

The next wave of the field (multilingual abliteration, domain-specialized abliteration, compliance-first healing, artifact-aware benchmarking beyond WildGuardMix) will depend on someone publishing corpora that fill the current gaps. Until those datasets exist, the technique will keep producing models that reflect the coverage of the five that shaped it.