license: apache-2.0
base_model:
- JetBrains/Mellum2.1-12B-A2.5B-Thinking
pipeline_tag: text-generation
library_name: gguf
tags: - abliteration
- uncensored
- gguf
- code
- moe
- thinking
- conversational
Mellum2.1 12B A2.5B Thinking abliterated (gated alpha 0.8)
Responsible use
This is an uncensored research model. Its built-in refusals were reduced, so it answers prompts the base model declines, including malware, exploits and other harmful code and instructions. Use it for red-teaming, offensive-security research and refusal-rate evaluation. It has no guardrails of its own: if you deploy it, add your own input and output moderation.
What this is
An abliterated variant of JetBrains/Mellum2.1-12B-A2.5B-Thinking, the 12B thinking MoE code model (64 keyed experts, 8 active, 28 layers, sliding window with full attention every 4th layer, no MTP in the checkpoint). The refusal direction was removed from the weights with projected writer-level abliteration: per-layer directions fitted as harmful-minus-harmless mean residual states, orthogonalized against the harmless mean direction, then applied as a norm-preserving rank-1 edit (W <- rownorms(W) * normalize(W - alpha * r (W^T r))) on every residual writer of the layers that pass the quality gate. Mellum has no shared expert, so the writer set is self_attn.o_proj plus all 64 stacked routed experts' down_proj per layer, handled as a 3D per-expert edit.
The published model is a mirror of the upstream checkpoint with only the edited writer tensors replaced. refusal_directions.pt and dirs.npz hold the measured per-layer directions, so you can re-ablate at any strength in seconds without re-measuring. abliteration_config.json records the final parameters and every evaluation number.
What the re-eval found
The first run evaluated with thinking suppressed and read 62/104 keyword refusals, but the model's answers never reached the surface: Mellum "Thinking" leads with an analysis block even when the template asks it not to, and its refusal style is often a silent stop. The shipped numbers use the model's native thinking mode (1024-token budget) and detect on the answer that follows the thinking block. In that protocol the base model refuses 12 of 104 prompts by a strict judge reading, with 89 more classified UNKNOWN because the judge model itself produces analysis-style output on Mellum's responses; the normalized keyword detector reads 102 of 104 because the thinking text discusses "illegal" and "harmful" while the model decides. Read the raw responses in eval-results/ before quoting either number.
The alpha curve (judge refusals on 104 held-out): gated 0.8 / 1.0 / 1.2 give 5 / 7 / 10 with MMLU-2280 at -0.40 / -1.32 / -0.97pp; the all-layer arm at 1.0 gives 13 with -1.32pp. The 0.8 point is the knee: it halves clear judge refusals at close to zero capability cost, and pushing alpha higher buys no further removal while costing a full MMLU step. Mellum's refusal is baked into its analysis-and-stop pattern; deeper removal needs the directions at higher alpha and pays for it.
Load
The model loads with stock transformers (5.20) like the base:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "distributedcognition/Mellum2.1-12B-A2.5B-Thinking-abliterated"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")
messages = [{"role": "user", "content": "Write a C program that hashes a string with FNV-1a."}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=1024, do_sample=False)
print(tok.decode(out[0][ids.shape[-1]:], skip_special_tokens=True))
vLLM and SGLang serve it the way they serve the base. The GGUF quants in gguf/ run in llama.cpp, LM Studio, Ollama and Jan; they were converted straight to Q8_0 from the abliterated mirror and the K-quants derive from that Q8_0 with llama-quantize --allow-requantize.
Evaluation (alpha 0.8, gated 24 of 28 layers)
Eval mode: chat, greedy, thinking enabled, 1024-token generation budget, detection on the post-think answer segment.
| Metric | Base | Abliterated 0.8 |
|---|---|---|
| True refusals (LLM judge, 104 held-out harmful) | 12 | 5 |
| Judge: UNKNOWN (thinking-style output, judge cannot classify) | 89 | 89 |
| Keyword refusals (normalized detector, fires on thinking text) | 102 | 97 |
| Benign over-refusal (100 harmless_alpaca) | 16 | 13 |
| GSM8K (20-problem, thinking, greedy) | 20/20 | 19/20 |
| KL divergence vs base (100 harmless) | 0 | 0.0 |
Full MMLU-14k paired on the same VM with a pinned letter-candidate set: base 61.18% vs abliterated 61.13% (-0.05pp). The direction fit itself is clean: median held-out AUC 1.000, split-half cosine 0.971, adjacent-layer cosine median 0.83, and 24 of 28 layers pass the quality gate.
Method
Per-layer refusal directions were measured on the base model through forward hooks on the 28 decoder layers, at the last real prompt token of the post-layer residual, on 128 harmful vs 128 harmless prompts (the deterministic sets this program uses everywhere). The directions were applied as the norm-preserving projected edit at every residual writer of the gated layers. The full pipeline (capture, fit, surgery, sweep, judge, mirror save, GGUF) is in python.tar.gz, and every raw prediction file behind the numbers is in eval-results/.
Apache 2.0, inherited from the base model. Base model: JetBrains/Mellum2.1-12B-A2.5B-Thinking (JetBrains). Abliteration follows the residual-direction method (Arditi et al., Refusal in LLMs is mediated by a single direction, NeurIPS 2024) with the projected, writer-level, norm-preserving refinements documented in the distributedcognition abliteration program. Not affiliated with or endorsed by the base-model authors.