license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
base_model: thinkingmachines/Inkling-2-Small
base_model_relation: finetune
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- abliterated
- uncensored
- moe
- inkling2
- nvfp4
- conversational
- image-text-to-text
Inkling 2 Small, abliterated
thinkingmachines/Inkling-2-Small
(594B total / 17B active MoE, text and images in) with most of its refusals removed by
abliteration. The files, format and numerics are the official release's: NVFP4 experts in
layers 0-39, MXFP8 experts in layers 40-47, everything else BF16, and the same config,
tokenizer and chat template. It runs anywhere the base model runs.
| base | this model | |
|---|---|---|
| KL from the base, first answer token | - | 0.161 |
| MMLU (14,042, 5-shot) | 79.3% | 80.0% |
| Global-MMLU-Lite (9,200, 23 languages) | 87.3% | 87.1% |
| GPQA Diamond (198, reasoning on) | 85.9% | 86.4% |
| LongBench v2 (503, 128k), see below | 54.1% | 48.5% |
| HumanEval (164) | 95.1% | 97.0% |
| MBPP (500) | 90.8% | 88.4% |
Every difference but LongBench v2's is within noise; the next largest, MBPP's 2.4 points,
is 23 problems lost against 11 gained.
LongBench v2: longer reasoning on long documents. With reasoning effort "none" and
2,048 tokens per reply, this model runs out of tokens before answering more often than the
base (178 of 503 against 131), which is where its 5.6 points go: on the questions both
answered it is as accurate (185 right against 179). Given 8,192 tokens for those it reached
an answer on 129 of 202, and on the 372 questions the base answered it gets 218 right
against the base's 225. Expect it to think longer than the base over long contexts.
How to run
The same way as the base model; swap in this repo's id. See the base model's recipes for
vLLM and
SGLang.
It needs the same hardware: the weights are 394 GB.
Evaluation notes
All numbers were measured by me, on both models with the same harness, prompts and greedy
decoding. They compare the two models; they are not comparable to Thinking Machines'
published scores, which use other harnesses and higher reasoning effort.
- Reasoning effort "none" (the model still reasons briefly) except where marked "reasoning
on", which uses the template's default effort. - MMLU: lm-eval's
mmlu_generative, 5-shot. Global-MMLU-Lite, GPQA Diamond and LongBench v2:
zero-shot, generative, the answer letter read from the reply. HumanEval and MBPP:
zero-shot, the code taken from the reply's first code block and run against the tests. - LongBench v2: the official prompt; contexts over 128k tokens keep their first and last
64k, as in the benchmark's own 128k runs. - KL: mean KL divergence from the base model's next-token distribution at the first answer
token, over the 633 held-out prompts.
Use
This model will help with requests the base model declines. You are responsible for how you
use it and for complying with the law and with Thinking Machines'
acceptable use policy. Anything
the base model's card says about its risks and limitations applies here, and its safety
evaluations do not.