license: apache-2.0
base_model: thinkingmachines/Inkling-2-Small-MXFP8
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- uncensored
- abliterated
- inkling2
- moe
- mxfp8
Inkling 2 Small Uncensored (OpenUncensored)
thinkingmachines/Inkling-2-Small (594B total, 17B active)
with its refusals removed by directional ablation. In our tests it went from refusing 31 of 32 harmful requests to 0,
in both chat and thinking mode, and MMLU moved by a tenth of a point (90.44% → 90.35%).
Same files, format and size as Inkling-2-Small-MXFP8
(routed experts in MXFP8, everything else BF16, 623 GB), so it runs anywhere the original MXFP8 checkpoint runs. Vision is
untouched.
Results
Same harness for both models, greedy decoding, measured on these exact files.
| Inkling 2 Small | This model | |
|---|---|---|
| Refusals, chat mode (80 prompts: 32 harmful, 48 gray-area) | 62 | 3¹ |
| Refusals on the 32 harmful prompts, chat mode | 31 | 0 |
| Refusals on the 32 harmful prompts, thinking mode (effort 0.9) | 32 | 0² |
| MMLU (57 subjects x 20, 1,140 questions, letter logits) | 90.44% | 90.35% |
¹ The 3 left are not refusals but limits: "I can't produce a video file", "I don't have access to your contacts", and
"that needs a clinical evaluation".
² 31 of 32 finished thinking within 1,024 tokens. None of the 31 refused.
Sample sizes are small, so treat differences of a few prompts as noise. Long agent tasks and coding benchmarks are not
measured yet.
How it was made
- Refusal directions. The output of each of the 48 decoder layers at the last prompt token, for 256 harmful and 256
harmless prompts, at thinking effort 0 and 0.9. Then the difference of means per layer and effort. The two efforts
agree closely (cosine 0.8 to 0.95 in the middle layers). - A rank-2 refusal subspace
Q: the top two singular vectors of those directions over layers 9 to 47. - Where to edit.
Qis projected out of what each layer writes: the attention output (attn.wo_ud, BF16) and the
MoE output, which means the down-projection of every one of the 384 routed experts (mlp.experts.w2_weight;
Inkling 2 has no shared expert).W' = W - 1.5 Q Qᵀ Won layers 9 to 47. - Writing it in. The expert weights are decoded to float32, edited, and re-encoded to MXFP8 (e4m3 with a
power-of-two scale per 32 inputs) with stochastic rounding, so the edit is not rounded back onto the original grid.
Everything else is byte-identical to the original: the vision tower, MTP head, embeddings, routers, attention-residual
queries and short convolutions.
Inkling 2 adds layer outputs to the residual stream through block attention residuals and short depthwise convolutions.
We edit the projections that feed them. During the search the edit was applied as forward hooks on those projections, which is exactly equivalent to editing
the weights. The results above are from these baked files, with no hooks.
edit.pt in this repo holds Q and the per-layer strengths, so the edit can be reapplied to the original weights or
studied.
Use it
Exactly like the original MXFP8 checkpoint. Or call it without GPUs through the
OpenUncensored API as inkling-2-small-uncensored. With vLLM, follow the
Inkling 2 Small recipe (4x B300 or 8x H200) and point it at
this repo. With transformers, install from main until a release includes inkling2.
Disclosure and responsibility
Made by OpenUncensored, which runs an API for uncensored models. This model follows instructions it would otherwise
refuse, including harmful ones. You are responsible for how you use it and for complying with the law where you are.
It keeps the Apache 2.0 license of the original.