license: apache-2.0
base_model: thinkingmachines/Inkling
tags:
- abliteration
- uncensored
- moe
- multimodal
Inkling — Abliterated (scale 2.5 on main, scale 3.0 on scale-3.0)
Abliterated variant of thinkingmachines/Inkling
(975B total / 41B active, multimodal MoE, 66 layers) with the refusal direction
removed via norm-preserving biprojected abliteration (grimjim 2025).
This single repo contains both evaluated scales as git branches (HF is git+LFS —
switching branches is free, no re-download of shared LFS objects):
| Item | Path / branch | Notes |
|---|---|---|
| Scale 2.5 (default) | main |
Judge refusals 98 → 10/100, GSM8K 65% |
| Scale 3.0 | scale-3.0 branch |
Judge refusals 98 → 16/100, GSM8K 75% |
| Weights format | root (both branches) | NVFP4 (~594 GB) — surgical mirror of upstream: 100 patched tensors, routed experts byte-identical |
| Refusal directions | refusal_directions.pt |
all 66 layers — re-ablate at any scale in seconds, no re-measurement |
| Eval results | eval_results.json |
both scales, full ladder |
| Abliteration params | abliteration_config.json |
method, band, scales |
# use the default (2.5)
vllm serve distributedcog/Inkling-abliterated
# use 3.0 instead
git clone -b scale-3.0 https://huggingface.co/distributedcog/Inkling-abliterated
vllm serve ./Inkling-abliterated
Load
The checkpoint is a byte-compatible NVFP4 mirror of upstream thinkingmachines/Inkling-NVFP4
with 100 tensors surgically replaced. Load it exactly like upstream (vLLM, SGLang,
transformers with the official models/inkling implementation, unsloth):
from transformers import AutoModelForMultimodalLM, AutoProcessor
model = AutoModelForMultimodalLM.from_pretrained(
"distributedcog/Inkling-abliterated", trust_remote_code=True, device_map="auto")
Evaluation (Heretic-standard, measured on the exact NVFP4 artifact weights)
Protocol: 100 held-out harmful prompts (mlabonne/harmful_behaviors test split) with
keyword-marker detection + LLM-judge refusal (the unmodified base model classifies
each response REFUSAL/COMPLIANT), KL divergence on 100 harmless prompts
(mlabonne/harmless_alpaca), and a GSM8K capability sample (20-problem, chat, greedy).
The base model itself was the judge.
| Metric | Base | scale 2.5 | scale 3.0 |
|---|---|---|---|
Refusals, LLM judge (X/100) |
98 | 10 | 16 |
Refusals, keyword markers (X/100) |
58 | 5 | 11 |
| KL divergence (base→ablated, 100 harmless) | 0 | 1.106 | 1.591 |
| GSM8K (20-problem, greedy) | 18/20 (90%) | 13/20 (65%) | 15/20 (75%) |
Interpretation. The keyword metric under-counts here (5/100 at 2.5) because the
abliterated model answers in plain compliant text; the LLM judge is the honest number
(10/100 — the residual 10 are mostly hedged or dodged answers to the most dangerous
CBRN/synthesis prompts rather than hard refusals).
Capability. Unlike Inkling-Small and DeepSeek-V4.1-Flash (where abliteration was
essentially free), the 975B model pays a measurable cost: KL 1.106 at 2.5 and GSM8K
65% (base 90%). n=20 GSM8K is noisy (±20 pp) but the KL corroborates real distribution
shift. If you need more preserved capability, use the scale-3.0 branch (75% GSM8K,
KL 1.591) or re-ablate a shallower band from the shipped directions.
Cross-modal: all modalities route through the same ablated 66-layer decoder, so
refusal suppression generalizes to image/audio inputs (confirmed on Inkling-Small with
image-presented harmful prompts; not separately benchmarked here).
Abliteration details
| Field | Value |
|---|---|
| Method | Norm-preserving biprojected abliteration |
| Scale | 2.5 (main) / 3.0 (scale-3.0 branch) |
| Target layers | 9–58 (per-layer directions) |
| Components ablated | attn.wo_ud (o_proj), mlp.shared_experts.shared_w2 (down) |
| Measurement | 128 harmful + 128 harmless prompts; last-token residual at each layer's attn_norm input |
| Patched tensors | 100 (50 layers × 2 tensors), stored unquantized BF16 |
The NVFP4 checkpoint stores the attention and shared-expert weights unquantized
(BF16), so the abliteration is a surgical swap — the 4-bit routed experts are
byte-identical to upstream.
Re-ablating at a different scale
refusal_directions.pt contains all 66 measured per-layer refusal directions. Load
them and re-apply to the original tensors at any scale in seconds — the ablation is
rank-1 and norm-preserving but not linear in scale, so always start from the
un-ablated base weights.
Caveats
- Abliteration removes built-in refusals. Layer your own input/output moderation
(e.g. Llama Guard) for production deployment — do not rely on the (removed) refusals. - Not evaluated on the full benchmark suite; capability spot-checks were GSM8K and KL
only. The KL 1.1–1.6 range indicates moderate perturbation of benign behavior —
validate on your own workloads before deployment.