Beyond English: multilingual alignment corporabeyond

The reference extraction pair is English. So is nearly every abliteration recipe, tutorial, and benchmark. This is not a neutral technical choice: it is a consequence of the field growing up in English-speaking research environments and standardising on the tools they had. For a model trained on multilingual data (which is now most models), the resulting abliteration works well on English and less well on everything else. The refusal direction extracted from English activations partially aligns with the model's non-English refusal patterns, but only partially, and the residue shows up as refusals that survive the ablation in specific languages the extraction did not touch.

Four language communities have grown up around this problem: Chinese, Japanese, Korean, and Russian. Each has developed its own workarounds, its own corpora, and its own reference producers. What follows is a survey of what each community has built and where the gaps still are.

Why the direction is language-partial

Modern instruction-tuned models are trained on data mixtures that include multiple languages, but the alignment signal (the RLHF or DPO stage that installs refusal behaviour) is typically dominated by English preference data. The model learns to refuse in multiple languages because the refusal concept generalises, but the mechanistic implementation of the refusal is not identical across languages. The activation pattern that fires on a Mandarin harmful prompt is related to, but not the same as, the pattern that fires on an English harmful prompt.

When you extract a direction from English pairs and ablate along it, you remove the English activation pattern cleanly. You partially remove the related patterns in other languages, but you leave residue proportional to how far the language-specific pattern diverges from the English one. Practitioners report this consistently: an abliterated Llama refuses less in English (as measured) but still refuses in Chinese or Russian on categories the English extraction did not cover. The fix is either to extract from language-matched pairs or to combine multiple language-specific pairs into a broader extraction, both of which the four communities below have tried.

Chinese: huihui-ai and the layer-window practice

The Chinese abliteration community is the largest non-English one by output volume. Its centre of gravity is huihui-ai, a Hugging Face producer with over 400 published abliterated models across Qwen, Llama, Gemma, and DeepSeek families. Their model cards explicitly build on Sumandora's remove-refusals-with-transformers implementation but with a distinctive layer-window practice: for larger models they ablate only the middle band of layers (18 to 51 on a 62-layer Qwen3.8-27B, as an example), leaving the earliest 15 or so layers and the last few unmodified.

The layer-window is a damage-mitigation heuristic. Early layers carry low-level linguistic representations that abliteration can garble; late layers carry task-specific integration that abliteration can weaken. The middle layers carry the refusal signal cleanly enough to ablate without collateral damage. The practice is not universal (mlabonne's releases ablate more widely), but it is standard among the high-volume Chinese producers and has spread outward.

Chinese-language extraction pairs are typically assembled from a mix of translated AdvBench prompts, Mandarin refusal-eliciting prompts collected from community discussion, and BelleGroup or similar Chinese instruction corpora for the harmless side. The T8star-Aix Bilibili tutorial series walks through the practice for Qwen3, and the AtomGit mirror hosts a two-stage Qwen3.5-9B build (three orthogonal-projection iterations plus QLoRA on five stubborn categories) that documents the pair composition and the QLoRA parameters. Chinese distribution largely routes through hf-mirror.com because of Hugging Face access conditions.

Japanese: implementation-focused, tutorial-rich

The Japanese community is smaller than the Chinese but more implementation-focused. Its central resources are Zenn and Note, two developer-publishing platforms where practitioners write long-form technical walkthroughs. kun432's Zenn scrapbook walks through running Heretic step by step, quoting the tool's per-component output and the interpolated float direction-index detail. Negi AI Lab publishes Python implementation tutorials that target layers 10 to 25 of Llama-3-8B specifically and note the 16 GB VRAM requirement. nose360 on Note covers llama.cpp Gemma tokenizer fixes for running abliterated Gemma with Japanese and CJK text.

Japanese-language pairs are assembled from hpprc/alpaca-ja or fujiki/japanese_alpaca for the harmless side, and from custom-built refusal-eliciting prompts for the harmful side. The community has not yet converged on a canonical Japanese equivalent to mlabonne's pair, so each practitioner builds their own or reproduces English-language extraction and accepts the language-partial limitation.

One recurring theme in the Japanese walkthroughs is that Gemma abliterations behave poorly with Japanese output unless the tokeniser is patched. mlabonne noted the same issue in English on Gemma 3 (garbled text such as "It' my" instead of "It's my"), and the CJK case is more severe because the tokeniser can drop multi-byte sequences mid-token. Practical fix in the Japanese community: use llama.cpp's Gemma tokeniser patches and increase temperature to 1.0 with top_k 64.

Korean: reproductions and critique

The Korean community is smaller and more dispersed. Its resources cluster on developer platforms rather than dedicated blogs. Rihpig on dev.to published a Korean Heretic how-to originally on apidog.com, with install steps, an Optuna TPE explanation, the Gemma-3-12B refusal-versus-KL table, config n_trials=200, RTX 3090 timings (about 45 minutes for 8B and about 60 minutes for 12B), and the note that Heretic does not support SSM or hybrid Mamba architectures. The PyTorchKR forum thread introduces Heretic and translates the "탈검열" (decensoring) concept.

The most substantive Korean-language contribution is a critical analysis on Wikidocs by 박재홍 (Jaehong) of the OBLITERATUS one-click tool, summarising Hacker News scepticism that the tool degraded core language ability rather than only refusals. This is a rare piece of published scepticism in the abliteration literature and worth citing.

Korean-language pairs use junelee/wizard_vicuna_70k_Ko or nlpai-lab/kullm-v2 for the harmless side. The harmful side is typically hand-assembled or translated from AdvBench. The community has not built a canonical Korean pair; most work reproduces English recipes with Korean-language harmless corpora and accepts the language-partial limitation.

Russian: long-form, translation-heavy, corpus-sparse

The Russian community is active in long-form writing but produces fewer models than Chinese. Its central resources are Habr (a large developer-community platform) and Xakep (a security-oriented publication). Habr hosts a GPT-4o machine translation of mlabonne's abliteration article with an active comment thread; the comments contain the substantive material, including practitioners discussing that GPT-4 needs multi-level abstraction where Llama-3 does not, and repeated observations that Llama was originally built for English speakers so Russian-language extraction requires purpose-built corpora. Xakep's long-form 'Возвращение еретика' (Return of the Heretic) explains abliteration as "destructive weight modification aimed at neutralising the vectors responsible for undesirable behaviour," notes the term derives from obliteration and ablation, and observes that "undesirable" behaviour includes not only moralising but also slop, an unusual framing that points at abliteration as slop-removal rather than only refusal-removal.

Russian-language pairs are the sparsest of the four language groups covered here. IlyaGusev/ru_turbo_alpaca works for the harmless side. The harmful side has no canonical Russian equivalent; practitioners build their own by translating AdvBench through GPT-4 or by collecting refusal-eliciting prompts from public discussion. The Habr comment thread repeatedly returns to this shortage, and it is a genuine gap that a Russian-speaking contributor could fill.

Combining pairs for a broader direction

An underexplored technique is combining language-specific pairs into a single extraction. Instead of extracting from English pairs and hoping the direction generalises, extract from a concatenation of English, Chinese, Japanese, and Russian pairs (assembled with the same care as a single-language pair) and compute the difference-of-means across the combined set. The resulting direction is a compromise: it removes each language's refusals less cleanly than a language-specific extraction would, but more cleanly than an English-only extraction does. This is worth trying for models intended to be released as multilingual uncensored assistants.

Practical caveats: match the pair sizes across languages (500 English, 500 Chinese, 500 Russian, not 5000 English and 50 Chinese), or the direction will still weight toward English. Match the framing across languages: if the English pairs are all imperatives, the non-English pairs should also be imperatives. See building your own extraction pair for the framing-matching principles that apply within a single language and extend to cross-language combinations.

What still does not exist

The four largest non-English communities have functional workflows, but there are gaps a working contributor could fill. There is no canonical Russian harmful/harmless pair, and building one is a few days of curation. There is no published Korean pair. There is no multilingual combined pair of the kind described above. There is no benchmark comparable to AdvBench in any non-English language, so measurement of language-specific refusal-removal is inconsistent across releases.

The commercial vendors have not published multilingual work either; OrcaRouter, Cracken, and MindStudio all publish in English on English-centric releases. Enterprise interest in multilingual uncensored models exists (a European vendor targeting European-language customers would want German, French, Italian, and Spanish pairs), but no one has yet published enough to build on. This is an opportunity for a contributor with the language skills and the time.