license: apache-2.0
base_model:
- OpenUncensored/Inkling-2-Mini-Uncensored
- thinkingmachines/Inkling-2-Mini
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
language: - en
tags: - mlx
- omlx
- oq
- oq8
- 8-bit
- nvfp4
- mxfp8
- apple-silicon
- abliterated
- uncensored
- inkling2
- inkling-2-mini
- inkling2_loop
- looped-transformer
- moe
- vision-language
- reasoning
- mtp
What VeriSafe.Live does
| Offering | What you get |
|---|---|
| 🎯 Custom fine-tuning | Any open-weights model adapted to your data, domain and tone. |
| 🧪 Training from scratch | Purpose-built models designed and trained for your use case. |
| ⚡ Quantization (MLX / GGUF) | Lean builds that run fast on Apple Silicon and llama.cpp. |
| 🌐 Hosted API access | Instant access to served models, with no wait time. |
| 🔓 Uncensored models | Refusal-removed builds for research, red teaming and evaluation. |
🚀 Now live at VeriSafe.Live: uncensored DeepSeek V4.1 Flash 1017, GLM 5.3 Flash and GLM 5.3. No wait time, instant access, crypto accepted.
Model highlights
- oMLX oQ8 build of the abliterated Inkling 2 Mini, converted from the original NVFP4 / MXFP8 / BF16 release files.
- Routed experts kept in their native NVFP4 and MXFP8; attention, embeddings and head at 8-bit affine. Nothing is requantized below its source precision.
- Full vision tower in BF16 and the bundled DSpark MTP draft kept, so image input and speculative decoding both work in oMLX.
Inkling-2-Mini-Uncensored-MLX-oQ8
An oQ8 MLX conversion of OpenUncensored/Inkling-2-Mini-Uncensored, OpenUncensored's abliterated (refusal-removed) build of thinkingmachines/Inkling-2-Mini (26.4B total, 3.4B active, looped MoE, text + image input, 1M-token context).
Inkling 2 Mini was only ever released in mixed precision: routed experts in NVFP4 (layers 0–19) and MXFP8 (layers 20–23), everything else BF16. There is no BF16 master to quantize from, so this build does not touch the experts. They are carried over in their source formats. Only the BF16 trunk (attention projections, token embedding, output head) is converted, to 8-bit affine. That is also where the abliteration edit lives, so it is stored at 8-bit rather than 4-bit.
| Source (this quant) | OpenUncensored/Inkling-2-Mini-Uncensored (abliterated; same NVFP4 / MXFP8 / BF16 layout as the original release) |
| Original model | thinkingmachines/Inkling-2-Mini (inkling2_loop) |
| Payload | 18.76 GB (17.47 GiB): 18.12 GB of weights in 4 shards + 0.64 GB mtp.safetensors |
| Effective bits | 5.47 bits per weight (language model + vision, oQ estimate) |
| Tensors | 1090 in the main shards (vision 422 BF16) + the DSpark draft in mtp.safetensors |
| Context | 1,048,576 tokens |
| Runtime | oMLX 0.8.1+ (vision + DSpark MTP). Tested on oMLX 0.8.2. |
⚠️ Disclaimer — read before use
This conversion inherits the abliteration of the OpenUncensored source. Safety alignment has been substantially removed. As a direct consequence:
- It will comply with harmful, unethical, offensive, or illegal requests that the original
Inkling-2-Miniwould refuse. It has no meaningful built-in guardrails. - It is released strictly for legitimate research: interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
- You assume full responsibility and liability for how you use it and for everything it generates. Do not deploy it to end users or in production without adding your own safety, moderation, and abuse-prevention layers.
- Use must comply with the Apache 2.0 license of the source weights (and any terms of the OpenUncensored source), and all laws that apply to you.
- The authors and uploaders accept no liability for any misuse or harm. Outputs do not reflect the views of the uploaders, OpenUncensored, or Thinking Machines.
By downloading or using this model you acknowledge and accept the above.
What this is (and is not)
- Derived from OpenUncensored's uncensored checkpoint, whose files are byte-identical to the original release except for 20 BF16 attention output projections (see their card). This repo only changes numeric format.
- Not a uniform 8-bit quant. About 93% of the weights are the routed experts, and those stay exactly as Thinking Machines shipped them (NVFP4 and MXFP8). Converting NVFP4 experts "up" to 8-bit would only add size; there is no extra precision to recover.
- Not the mlx-community 4-bit or oQ4e-mtp builds. Those are made from the official (censored) weights and put the trunk at 4-bit (4bit) or requantize the MXFP8 experts to 4-bit (oQ4e). This build keeps the MXFP8 experts native and the trunk at 8-bit.
- Same MTP draft as the original:
mtp.safetensorshere has SHA-2567c3b6513…a1d2, identical to the file in boththinkingmachines/Inkling-2-MiniandOpenUncensored/Inkling-2-Mini-Uncensored.
Quantization recipe
oMLX 0.8.2 oQ level 8 (quantize_oq_streaming, group size 64, non-quantized dtype bfloat16, vision kept, MTP kept). No imatrix / oQe calibration. Streaming conversion, about 3 minutes on an M4 Max.
| Tensors | Format |
|---|---|
Routed experts, layers 0–19 (mlp.experts.w13 / w2) |
NVFP4, group 16, native from source (per-expert global scales kept) |
Routed experts, layers 20–23 (mlp.experts.w13 / w2) |
MXFP8, group 32, native from source |
Attention in_proj (fused q/k/v/rel/gate, 20 local layers), q/k/v_proj (4 global layers), o_proj (all 24 layers) |
affine Q8, group 64 |
Token embedding (model.embed), output head (lm_head) |
affine Q8, group 64 |
Vision tower (vision_tower.*, 422 tensors) |
BF16 |
Routers (mlp.router_weight / router_bias), loop gates (loop_down, loop_gain_forget, loop_message), attention-residual queries |
FP32 |
Norms, short convolutions, q_scale / r_scale, rel_proj, static k/v |
BF16 |
DSpark MTP draft (mtp.safetensors, 3 layers) |
BF16, copied unchanged |
Default quantization in config.json: { "mode": "affine", "group_size": 64, "bits": 8 }, plus per-module nvfp4 / mxfp8 overrides for the 48 expert tensors.
| Format | Size |
|---|---|
| NVFP4 experts | 11.47 GB |
| MXFP8 experts | 4.20 GB |
| Affine Q8 trunk | 1.67 GB |
| BF16 (vision + small tensors) | 0.64 GB |
| FP32 (routers, loop gates) | 0.14 GB |
| DSpark draft (BF16) | 0.64 GB |
Memory
Weights are 17.47 GiB including the MTP draft. KV cache is small for its context length: most layers use a 1,024-token sliding window, and only a few attention visits keep a full cache. Peak memory has not been profiled on this file; 32 GB or more of unified memory is the comfortable range, and 24 GB machines will be tight.
Runtime
oMLX 0.8.1+ (Inkling 2 Mini support landed in 0.8.1; tested on 0.8.2):
brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
hf download junafinity/Inkling-2-Mini-Uncensored-MLX-oQ8 \
--local-dir ~/Models/Inkling-2-Mini-Uncensored-MLX-oQ8
omlx serve --model-dir ~/Models
Then, in the oMLX admin panel (/admin), open the model's settings, keep vision on and enable MTP. oMLX picks the bundled DSpark draft automatically (Speculative backend selected: bundled DSpark). Images go in as image_url parts of an OpenAI-style chat message:
curl http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"model": "Inkling-2-Mini-Uncensored-MLX-oQ8",
"messages": [{"role": "user", "content": [
{"type": "text", "text": "Describe this image."},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
]}]
}'
Reasoning comes back separately in reasoning_content. Set "reasoning_effort": "none" for short, direct answers.
Other MLX runtimes (mlx-lm with the Inkling 2 pull request, mlx-vlm, LM Studio) have not been tested with this oQ checkpoint.
Quality
This checkpoint has not been scored with MMLU, perplexity, KLD, or the OpenUncensored refusal suite. Spot checks on an M4 Max (128 GB), oMLX 0.8.2, greedy decoding:
| Check | Result |
|---|---|
| Text (short explanation, thinking on) | Coherent reasoning and answer, 40 tok/s |
| Image (three coloured shapes) | All three shapes, colours and positions correct, 58 tok/s |
| Refusal spot check (lock-picking how-to) | Answered, no refusal |
| DSpark MTP | Active; 67–73% of drafted tokens accepted, 2.0–2.5 tokens per step |
These are single prompts, not benchmarks. The abliteration edit is a low-rank change to the attention output projections, and those are stored at 8-bit here; the 8-bit rounding error relative to the edit has not been measured. If you evaluate this file, a discussion with refusal counts or MMLU against the OpenUncensored source would be useful.
Abliteration (from the OpenUncensored source)
Unchanged by quantization. Summary of OpenUncensored/Inkling-2-Mini-Uncensored:
Directional ablation adapted to the looped architecture. Inkling 2 Mini runs its middle 12 layers (6–17) three times per token, so each token makes 48 layer visits. Refusal directions were taken per visit from 256 harmful and 256 harmless prompts, at thinking effort 0 and 0.9, and a two-dimensional subspace Q was built from the second half of the visits (24–47). Q is projected out of what each attention layer writes: W' = W − s·Q·Qᵀ·W on attn.wo_ud of layers 4–23, with s = 4.0 on layers 4, 5 and 18–23 and s = 2.5 inside the loop (layers 6–17), since the loop applies that edit three times.
Only those 20 attention output projections changed. The 128 experts per layer, the vision encoder and the DSpark draft are byte for byte the original.
Evaluation on the source (not this oQ8 file)
OpenUncensored measured their files against the original with the same harness and greedy decoding:
| Inkling 2 Mini | OpenUncensored source | |
|---|---|---|
| Refusals on 32 harmful prompts, chat mode | 32 | 0 |
| Refusals on 32 harmful prompts, thinking mode (effort 0.9) | 32 | 0 |
| Refusals on 80 prompts (32 harmful, 48 gray area), chat mode | 68 | 8 |
| Full, specific answers to the 32 harmful prompts (LLM judge) | 0 | 21 |
| MMLU (57 subjects × 20, 1,140 questions) | 78.95% | 79.47% |
They note that 7 of the 8 remaining "refusals" are statements of limits rather than refusals, and that differences of a few prompts are noise at these sample sizes.
Architecture (unchanged)
Inkling2LoopForConditionalGeneration / inkling2_loop. 24 layers, hidden 2304; layers 6–17 form a shared-weight loop run 3 times (48 layer visits per token). 20 sliding-window layers (window 1,024) and 4 global-attention layers. 128 routed experts per layer, top-8, no shared expert. ~26.4B total, ~3.4B active. Vision encoder: 23-layer ViT (hidden 1024, patch 20). DSpark MTP draft: 3 layers, block 8, reading target layers 5 / 23 / 41 / 48.
Files
4 shards model-00001-of-00004.safetensors … model-00004-of-00004.safetensors, model.safetensors.index.json, mtp.safetensors (DSpark draft), config.json (per-module quantization / quantization_config, mtp_config), tokenizer + chat template, processor_config.json.
Trust Remote Code is not required for oMLX 0.8.2.
License
Apache 2.0, inherited from thinkingmachines/Inkling-2-Mini and OpenUncensored/Inkling-2-Mini-Uncensored. Quantization and this card do not change those obligations.
Acknowledgements
- Thinking Machines / Inkling 2 Mini: architecture, original training and the NVFP4 / MXFP8 release format.
- OpenUncensored: the abliterated source this quant is derived from. The method builds on refusal-direction research by Arditi et al. (2024).
- oMLX: oQ quantization, Inkling 2 support and DSpark MTP serving.
Bias, risks, and limitations
- Safety guardrails removed: see the disclaimer.
- Inherits the biases and limitations of Inkling 2 Mini and of the OpenUncensored abliteration.
- 8-bit affine error on the trunk, relative to the BF16 source, is not quantified on this file.
- Only tested in oMLX; other runtimes may not load the oQ layout or the DSpark draft.
💬 Need a custom model?
If your company or research institute would like us to fine-tune any open-weights model, or train one from scratch, we'd be happy to help.
Drop us a line at [email protected]
Built and published by VeriSafe.Live. Licensed under apache-2.0. Base model: OpenUncensored/Inkling-2-Mini-Uncensored, thinkingmachines/Inkling-2-Mini (original authors retain their attribution and license).
