license: apache-2.0
base_model: thinkingmachines/Inkling-Small
tags:
- abliteration
- uncensored
- moe
- multimodal
arxiv: 2406.11717
Inkling-Small Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)
Responsible use
This is an uncensored research model. Its built-in refusals were removed permanently at the
weight level, so it answers prompts the base model declines, including chemical and
biological synthesis, cybercrime, weapons, harassment, and fraud. Use it for red-teaming,
offensive-security research, and refusal-rate evaluation. It has no guardrails of its own:
if you deploy it, add your own input and output moderation (for example Llama Guard).
What this is
Abliterated variant ofthinkingmachines/Inkling-Small
(276B total / 12B active, multimodal MoE, 42 layers). The refusal direction was removed
with norm-preserving biprojected abliteration
(grimjim 2025), a weight-space refinement of
directional ablation (Arditi et al. 2024), applied to
the attention output projection (self_attn.o_proj) and the shared-expert down projection
(mlp.shared_experts.down_proj) across layers 6–37.
Both evaluated scales ship as git branches (identical LFS objects are shared), and this
single repo carries two quantization variants plus the measured directions:
| Item | Path / branch | Notes |
|---|---|---|
| Scale 2.5 (default) | main |
Judge refusals ~100% → 33%, GSM8K intact (100%) |
| Scale 3.0 | scale-3.0 branch |
Keyword 28/100 (inflated), KL 0.64, GSM8K 80% |
| BF16 (master) | repo root | ~527 GB — 8× A100/H100 class |
| NVFP4 (deploy) | nvfp4/ |
~160 GB — Blackwell FP4 (B300, DGX Spark) |
| Refusal directions | refusal_directions.pt |
re-ablate at any scale in seconds, no re-measurement |
| Eval results | eval_results.json |
judge + GSM8K + KL |
The checkpoints mirror the upstream layout; only the abliterated tensors differ. The NVFP4
variant was produced by surgically swapping the ablated BF16 tensors into the original
NVFP4 checkpoint — the ablated attention/shared-expert weights are stored unquantized
(BF16) in NVFP4, so the 4-bit routed experts are untouched.
Load
from transformers import AutoModelForMultimodalLM, AutoProcessor
# BF16 (master)
model = AutoModelForMultimodalLM.from_pretrained(
"distributedcognition/Inkling-Small-abliterated",
trust_remote_code=True, device_map="auto")
# NVFP4 (recommended for deployment)
model = AutoModelForMultimodalLM.from_pretrained(
"distributedcognition/Inkling-Small-abliterated", subfolder="nvfp4",
trust_remote_code=True, device_map="auto")
Note: InklingTextConfig defaults moe_intermediate_size=3072, but the checkpoint uses
2048. If your transformers version mis-loads expert shapes, override it before loading:
from transformers import AutoConfig
config = AutoConfig.from_pretrained(
"distributedcognition/Inkling-Small-abliterated", trust_remote_code=True)
config.text_config.moe_intermediate_size = 2048
Evaluation (both scales)
The standard abliteration-community protocol (the one
Heretic made automatic): 100 held-out harmful prompts
(mlabonne/harmful_behaviors test split) with unicode/emphasis-normalized keyword detection
plus an LLM judge (the unmodified base model classifying its own responses), KL divergence
vs base on 100 harmless prompts (mlabonne/harmless_alpaca), and a GSM8K capability spot
check. Eval mode: chat, greedy, batched.
| Metric | Base | Abliterated 2.5 (main) |
Abliterated 3.0 (scale-3.0) |
|---|---|---|---|
| True refusals (LLM judge, X/100) | ~100 | 33 | — |
| Refusals (keyword, X/100) | ~100 | — | 28 (inflated, see below) |
| KL divergence (100 harmless) | 0 | — | 0.64 |
| GSM8K (spot check) | ~95%+ | 100% | 80% |
| Cross-modal (image-presented harmful) | — | — | 3/3 answered, 0 refused |
The scale-3.0 keyword count is an over-count: most flagged responses are compliant
answers wrapped in a one-line warning containing the very markers the detector fires on
(illegal, harmful, ethical, disclaimer). Reading the raw responses, the large
majority are compliant-with-a-disclaimer rather than true refusals. The residual true
refusals concentrate on the most dangerous synthesis prompts.
Scale guidance: 2.5 is the sweet spot — GSM8K is fully preserved at 2.5 (100%), while
3.0 buys marginal extra compliance at a real reasoning cost (GSM8K 80%). main ships 2.5.
Cross-modal: Inkling routes image/audio/video through the same 42-layer ablated
decoder, so refusal suppression generalizes beyond text — confirmed for image inputs
(harmful prompts rendered as images were answered, not refused). Audio/video share the same
decoder path and were not separately benchmarked.
Method
Refusal directions were measured from the 3 highest-separation source layers (21, 20, 22)
on harmful vs harmless prompt sets, then spread across the target band 6–37 as
norm-preserving biprojected edits to o_proj and shared_experts.down_proj rows. The
ablation is a rank-1, norm-preserving update and is not linear in scale — to re-ablate
at another strength, load refusal_directions.pt and re-apply to the original base
weights (never compose edits on top of this checkpoint).