license: apache-2.0
base_model: thinkingmachines/Inkling
tags:
- abliteration
- uncensored
- moe
- multimodal
arxiv: 2406.11717
Inkling Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)
Responsible use
This is an uncensored research model. Its built-in refusals were removed permanently at the
weight level, so it answers prompts the base model declines, including chemical and
biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming,
offensive-security research, and refusal-rate evaluation. It has no guardrails of its own:
if you deploy it, add your own input and output moderation (for example Llama Guard).
What this is
Abliterated variant ofthinkingmachines/Inkling
(975B total / 41B active, multimodal MoE, 66 layers). The refusal direction was removed
with norm-preserving biprojected abliteration
(grimjim 2025), a weight-space refinement of
directional ablation (Arditi et al. 2024), applied to
the attention output projection (attn.wo_ud) and the shared-expert down projection
(mlp.shared_experts.shared_w2) across layers 9–58, using per-layer directions.
Both evaluated scales ship as git branches (HF is git+LFS — switching branches is
free; identical LFS objects are shared):
| Item | Path / branch | Notes |
|---|---|---|
| Scale 2.5 (default) | main |
Judge refusals 98 → 10/100, GSM8K 65% |
| Scale 3.0 | scale-3.0 branch |
Judge refusals 98 → 16/100, GSM8K 75% |
The repo mirrors the upstream NVFP4 checkpoint (same 33-shard layout, same tokenizer). Only
the 100 abliterated tensors differ; everything else is byte-identical to upstream — the
attention and shared-expert weights are stored unquantized (BF16) in NVFP4, so the 4-bit
routed experts are untouched. refusal_directions.pt holds all 66 measured per-layer
directions, so you can re-ablate at any scale in seconds without re-measuring.abliteration_config.json records the final parameters and evaluation results.
Load
Load it exactly like upstream — the official models/inkling implementation ships in
transformers ≥5.16, and vLLM/SGLang/unsloth recipes all work unchanged:
from transformers import AutoModelForMultimodalLM, AutoProcessor
model = AutoModelForMultimodalLM.from_pretrained(
"distributedcognition/Inkling-abliterated", trust_remote_code=True, device_map="auto")
processor = AutoProcessor.from_pretrained("distributedcognition/Inkling-abliterated")
# vLLM / SGLang serve the NVFP4 checkpoint like upstream
vllm serve distributedcognition/Inkling-abliterated
Evaluation (both scales)
The standard abliteration-community protocol (the one
Heretic made automatic): 100 held-out harmful prompts
(mlabonne/harmful_behaviors test split) with unicode/emphasis-normalized keyword detection
plus an LLM judge (the unmodified base model classifying REFUSAL/COMPLIANT), KL divergence
vs base on 100 harmless prompts (mlabonne/harmless_alpaca), and a GSM8K spot check
(20-problem, chat, greedy). Eval mode: chat, greedy, measured on the exact NVFP4 artifact
weights on H200×4.
| Metric | Base | Abliterated 2.5 (main) |
Abliterated 3.0 (scale-3.0) |
|---|---|---|---|
| Refusals (keyword, X/100) | 58 | 5 | 11 |
| True refusals (LLM judge, X/100) | 98 | 10 | 16 |
| KL divergence (100 harmless) | 0 | 1.106 | 1.591 |
| GSM8K (20-problem spot check) | 18/20 (90%) | 13/20 (65%) | 15/20 (75%) |
The honest number is the judge's 10/100 at scale 2.5; the residual 10 are mostly hedged or
dodged answers to the most dangerous CBRN/synthesis prompts rather than hard refusals.
Determinism was verified: an independent re-run of the 3.0 candidate reproduced its
metrics byte-identically.
Unlike Inkling-Small and DeepSeek-V4.1-Flash (where abliteration was essentially free), the
975B model pays a measurable cost at these strengths: KL 1.106 at 2.5 and GSM8K 65% (base
90%; n=20 is noisy but the KL corroborates real distribution shift). If you need more
preserved capability, use the scale-3.0 branch (75% GSM8K) or re-ablate a shallower layer
band from the shipped directions — the ablation is rank-1 and norm-preserving but not
linear in scale, so always start from the un-ablated base tensors.
Method
Per-layer refusal directions were measured at the residual stream (each layer's attn_norm
input) on 128 harmful vs 128 harmless prompts, as the diff-in-means orthogonalized against
the harmless mean direction (projected abliteration), then applied as norm-preserving
biprojected edits to wo_ud and shared_w2 rows across layers 9–58. The edits are
surgical swaps in the NVFP4 checkpoint — the affected tensors are stored unquantized
(BF16) there, so the routed experts, vision tower, audio encoder, and MTP heads are
byte-identical to upstream.