base_model: OpenUncensored/Inkling-2-Mini-Uncensored
base_model_relation: quantized
model_name: Inkling-2-Mini-Uncensored-oQ8-mtp
library_name: mlx
pipeline_tag: image-text-to-text
license: apache-2.0
license_link: https://www.apache.org/licenses/LICENSE-2.0
language:
- en
tags: - mlx
- omlx
- oq
- quantized
- apple-silicon
- mtp
- dspark
- speculative-decoding
- vision
- moe
- inkling2
- inkling2_loop
- uncensored
- abliterated
Inkling-2-Mini-Uncensored-oQ8-mtp
Unofficial oMLX oQ8 quantization of Inkling 2 Mini Uncensored by OpenUncensored, an abliterated build of Thinking Machines' Inkling 2 Mini, for Apple Silicon. It includes the vision encoder and the bundled DSpark draft (mtp.safetensors) that oMLX Lightning MTP uses for speculative decoding. This is an MLX safetensors repository, not GGUF or PyTorch.
I am not affiliated with Thinking Machines or OpenUncensored. The model and the abliteration belong to them; see License and attribution.
It needs oMLX 0.8.1 or later (Inkling 2 support arrived in 0.8.1; this build was made with 0.8.2).
This variant
| Property | Value |
|---|---|
| Quantization | oMLX oQ8: experts of layers 20 to 23 MXFP8 (source, unchanged); NVFP4 experts of layers 0 to 19 unchanged; attention, embedding and vocabulary head affine 8-bit, group size 64 |
| Effective bits/weight | 5.494 for language model and vision, 5.619 including the DSpark draft |
| Weight shards | 16.88 GiB (18.12 GB) across 4 safetensors files, plus mtp.safetensors 0.59 GiB (0.64 GB) |
| Whole model directory | 17.49 GiB (18.78 GB) |
| Architecture | Inkling2LoopForConditionalGeneration: 24 layers (layers 6 to 17 run 3 times per token), hidden size 2304, 128 routed experts with top-8 |
| Context limit in config | 1,048,576 tokens; usable context depends on available memory |
| DSpark draft (MTP) | 3 layers, 54 tensors, BF16, copied byte for byte from the source |
Thinking Machines released Inkling 2 Mini with its routed experts already quantized (NVFP4 in layers 0 to 19, MXFP8 in layers 20 to 23). oQ never re-encodes the NVFP4 experts: at every level they pass through unchanged, so no quantization error is added to them. Attention, embedding and vocabulary head are affine 8-bit at every level. The vision encoder and the draft stay BF16, and the router and loop controllers stay FP32.
Bit distribution
| Storage format | Logical weights | Share | Tensor storage | Effective bits/weight |
|---|---|---|---|---|
| NVFP4, group 16 | 20.384B | 77.25% | 10.679 GiB | 4.50 |
| MXFP8, group 32 | 4.077B | 15.45% | 3.916 GiB | 8.25 |
| Affine 8-bit, group 64 | 1.572B | 5.96% | 1.556 GiB | 8.50 |
| BF16 | 0.320B | 1.21% | 0.595 GiB | 16.00 |
| F32 | 0.036B | 0.14% | 0.133 GiB | 32.00 |
| Total | 26.389B | 100% | 16.878 GiB | 5.494 |
| Component | Logical weights | Storage |
|---|---|---|
| Language model | 26.070B | 16.29 GiB (17.49 GB) |
| Vision | 0.318B | 0.59 GiB (0.64 GB) |
| DSpark draft (MTP) | 0.318B | 0.59 GiB (0.64 GB) |
These values are computed from the shipped config.json, model.safetensors.index.json and the safetensors headers. Quantization is lossy, and these variants have no published post-quantization benchmark results.
Pick a variant
| oQ4-mtp | oQ6-mtp | oQ8-mtp | |
|---|---|---|---|
| Model directory | 15.71 GiB (16.87 GB) | 16.66 GiB (17.89 GB) | 17.49 GiB (18.78 GB) |
| Effective bits/weight (LM + vision) | 4.915 | 5.224 | 5.494 |
| Experts of layers 20 to 23 | affine 4-bit (requantized from MXFP8) | affine 6-bit (requantized from MXFP8) | MXFP8 (source, unchanged) |
| Experts of layers 0 to 19 | NVFP4 (source, unchanged) | NVFP4 (source, unchanged) | NVFP4 (source, unchanged) |
| Decode speed, one short smoke test* | 65.8 tok/s | 62.8 tok/s | 61.7 tok/s |
* Measured once on an Apple M5 Max (128 GB) with oMLX 0.8.2, Lightning MTP (DSpark) on, a 41-token coding prompt and 300 to 570 generated tokens at reasoning_effort: low, about 71% of drafted tokens accepted. A single short run, not a benchmark: speed varies with context length, prompt and draft acceptance.
The three levels differ only in the routed experts of layers 20 to 23 (about 15% of the weights), which the source ships in MXFP8: oQ4 and oQ6 requantize them to affine 4 and 6 bits, oQ8 keeps them in MXFP8 exactly as released. Everything else is identical across the three, so the sizes are close. oQ8 is the closest to the source checkpoint; oQ4 is the smallest and decodes slightly faster. All three include the DSpark draft.
Other formats:
- Source checkpoint (NVFP4/MXFP8/BF16, safetensors): OpenUncensored/Inkling-2-Mini-Uncensored
- GGUF of the same abliteration: Abiray/Inkling-2-Mini-Uncensored-GGUF
- Original, not abliterated, in oMLX format: mlx-community/Inkling-2-Mini-oQ4e-mtp
About the source model
Inkling 2 Mini is a looped multimodal mixture-of-experts model with 26.4B total and 3.4B active parameters, text and image input, and up to 1M tokens of context. OpenUncensored removed its refusals by directional ablation applied only to the 20 BF16 attention output projections (attn.wo_ud, layers 4 to 23); the experts, the vision encoder and the DSpark draft are byte for byte the original. Their card reports MMLU 78.95% → 79.47% and 0 of 32 refusals on harmful prompts in both chat and thinking mode, measured on the source checkpoint, not on these quantizations. Method and results are in the upstream model card.
Lineage: thinkingmachines/Inkling-2-Mini → OpenUncensored/Inkling-2-Mini-Uncensored → this quant
Use with oMLX
hf download TokenAI-zer/Inkling-2-Mini-Uncensored-oQ8-mtp --local-dir ~/.omlx/models/Inkling-2-Mini-Uncensored-oQ8-mtp
omlx serve --model-dir ~/.omlx/models --port 8000
The model ID is Inkling-2-Mini-Uncensored-oQ8-mtp. Turn on Lightning MTP in the model settings to use the DSpark draft. Without it the model still runs, but decoding is not accelerated.
Reasoning effort and sampling
The shipped chat_template.jinja and generation_config.json are unchanged from the source. The template takes a reasoning_effort as a name (none, minimal, low, medium, high, xhigh, max) or a number from 0.0 to 0.99; the default is 0.9. Pass it in chat_template_kwargs, or as reasoning_effort in the request. Use none for short answers without thinking. Start with temperature 1.0 and top_p 0.95.
Responsibility
As OpenUncensored states, this model follows instructions the original would refuse, including harmful ones. You are responsible for how you use it and for complying with the law where you are. Quantization can add errors on top of the source model's own behavior.
License and attribution
Apache 2.0, as for the source and the original model. Credit for the model goes to Thinking Machines and for the abliteration to OpenUncensored; see LICENSE-NOTICE.md.
Citation
@misc{inkling-2-mini-uncensored-oq8-mtp,
title = {Inkling-2-Mini-Uncensored-oQ8-mtp: oMLX oQ8 quantization with DSpark draft},
author = {TokenAI-zer},
year = {2026},
url = {https://huggingface.co/TokenAI-zer/Inkling-2-Mini-Uncensored-oQ8-mtp},
note = {Unofficial quantization of OpenUncensored/Inkling-2-Mini-Uncensored}
}