← back to catalog · registered 2026-11-21 22:04

arelius/Inkling-2-Mini-Uncensored-Q2-3090-GGUF

arelius GGUF MoE second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/arelius%2FInkling-2-Mini-Uncensored-Q2-3090-GGUF"
Response includes
  • classification m-uncensored
  • files 4
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M-U
Primary method

Uncensored (method unknown)

No other method signals detected in this model.
Confidence
LOW
Why this label 3 signals
Weak or ambiguous signals. Best guess based on catalog patterns; treat as tentative and check the evidence below.
  • 'uncensored' in name/tags but no 'abliterated' marker
  • method not identifiable from author declaration alone
  • may be DPO fine-tune, prompt engineering, or unknown technique
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
0
Likes
1
Model age
today
created 2026-11-21
Downloads over time
Now 0 → from 0 ↑ 0%
0011 0 on Nov 21 0 on Nov 22 Nov
Nov 21 → Nov 22 · 2 snapshots · spans 1 day

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Tags
gguf llama.cpp inkling2 moe 2-bit uncensored rtx-3090 text-generation base_model:AtomicChat/Inkling-2-Mini-Uncensored-GGUF base_model:quantized:AtomicChat/Inkling-2-Mini-Uncensored-GGUF license:apache-2.0 endpoints_compatible

Related

Total size
9.26 GB
Files
4
Quantizations
1
Registered
2026-11-21 22:04
Last updated on HF
2026-11-22 00:18

Files by quantization

Auxiliary files 4 files 9.26 GB
Inkling-2-Mini-Uncensored-Q2K-Q2_0.gguf 9.26 GB 76aaaac9 download
LICENSE 11.1 KB d6456956 download
README.md 5.50 KB 1c74c182 download
.gitattributes 1.56 KB a14eca34 download

README current version from Hugging Face


license: apache-2.0
base_model: AtomicChat/Inkling-2-Mini-Uncensored-GGUF
base_model_relation: quantized
pipeline_tag: text-generation
tags:

  • gguf
  • llama.cpp
  • inkling2
  • moe
  • 2-bit
  • uncensored
  • rtx-3090

Inkling 2 Mini Uncensored, 2-bit experts for one RTX 3090

A speed-oriented GGUF of AtomicChat's uncensored build of
thinkingmachines/Inkling-2-Mini. It uses the same
2-bit expert layout as our regular build and
decodes at about 200 tokens per second on a single RTX 3090 with our llama.cpp fork.

What "uncensored" means here: this is AtomicChat's edit, not ours. They describe it as a rank-1
LoRA on the 49 places that write into the residual stream (the attention output and routed experts of all
24 layers, plus the token embedding). In each layer it removes that layer's own refusal direction. See their card,
AtomicChat/Inkling-2-Mini-Uncensored-GGUF,
for what the edit does and how they evaluated it. The model will follow requests that the original declines.
You are responsible for how you use it.

Write-up: Looping on a 3090 (link coming soon) · Code:
csaben/llama.cpp, branch inkling-3090

File

File Size SHA-256
Inkling-2-Mini-Uncensored-Q2K-Q2_0.gguf 9.25 GiB 76aaaac94aa25fa517a34e16b2213296f3c0f209d70c32e9949989625ccc0c5b

Text only. No vision projector is included.

How it was made

The source is AtomicChat's baked Inkling-2-Mini-Uncensored-AD-Q4_K.gguf, at repository revision
40886ab636cabb4decf4998224f790b462ab048e. It was requantized with the same per-tensor recipe as the regular
build (all 704 tensor types match):

Tensors Type
Routed experts, layers 0–19: ffn_gate_up_exps Q2_K
Routed experts, layers 0–19: ffn_down_exps Q2_0, calibrated
Routed experts, layers 20–23 IQ4_NL
Attention projections and output head IQ4_NL
Token embeddings Q8_0
Norms, router, loop controllers, biases F32

Because the source was already a 4-bit checkpoint, this file has passed through two rounds of quantization. The
regular build was quantized directly from Q8_0. The calibrated Q2_0 encoder is in commit
704a7da. The on-disk layout is standard Q2_0.

Measured on one RTX 3090

These were measured with the inkling-3090 fork and the flags below. Perplexity uses the fork's fixed text
(perf/ppl.txt) and measures language-modeling quality, not refusal behavior.

Model Decode tok/s, empty context at 1K context Prefill PPL (8×2048) Decode PPL (2×1024, ubatch 1)
Original Inkling 2 Mini, ggml-org IQ4_NL (unfused reference) — — 2.9401 4.0660
Regular 2-bit build 208.7 193.2 3.0530 4.1945
This model 207.9 193.2 3.0571 4.1918

Speed drops as the context fills, because every one of the 48 layer passes rereads the attention cache. Expect
about 120–140 tok/s at the 16K–48K contexts coding agents use. The regular build's card has the full table.

Run it

git clone -b inkling-3090 https://github.com/csaben/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF
cmake --build build -j --target llama-server

export LLAMA_INKLING_CTRL_F16=1 IK_BATCH_MESSAGES=1 IK_SEGMENTED_IQ4=1 IK_SEGMENTED_Q6=1 IK_MIX_SPECIALIZE=1 IK_CTRL_APPLY8=1
build/bin/llama-server -m Inkling-2-Mini-Uncensored-Q2K-Q2_0.gguf -ngl 99 -fa on -ctk q8_0 -ctv q8_0 \
  -c 65536 -np 1 --backend-sampling --port 8031

Stock llama.cpp with Inkling 2 Mini support
(#32694) also loads the file, but much more slowly.

Limitations

  • Two quantization rounds, with perplexity about 4% above the 4-bit original. Check it on your own tasks.
  • Refusal behavior comes from AtomicChat's edit. We did not evaluate it.
  • Text only. The speed depends on the fork's Ampere-tuned (sm_86) fused kernels.

License and credits

Apache-2.0, matching the base model and AtomicChat's build. This is a quantized derivative. We made no changes
beyond quantization.

Citation

@misc{thinkingmachines2026inkling2mini,
  author = {{Thinking Machines Lab}},
  title  = {Inkling 2 Mini},
  year   = {2026},
  url    = {https://huggingface.co/thinkingmachines/Inkling-2-Mini},
  note   = {Announcement: https://thinkingmachines.ai/news/inkling-2-mini/}
}

@misc{atomicchat2026inkling2miniuncensored,
  author = {{AtomicChat}},
  title  = {Inkling-2-Mini-Uncensored-GGUF},
  year   = {2026},
  url    = {https://huggingface.co/AtomicChat/Inkling-2-Mini-Uncensored-GGUF}
}

@misc{saben2026looping3090,
  author = {Clark Saben},
  title  = {Looping on a 3090},
  year   = {2026},
  url    = {https://github.com/csaben/llama.cpp/tree/inkling-3090},
  note   = {Write-up link to follow}
}

README history 5 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-22 Link the published write-up b5e8d35 6.1 KB
    Loading...
  2. 2026-11-21 Launcher and release v2 (faster long context) ee0c2cf 6.1 KB
    Loading...
  3. 2026-11-21 Launcher v1.1; link the Q6_K quality build 5f6c374 6.1 KB
    Loading...
  4. 2026-11-21 Add the one-command uvx quick start 6853084 6.1 KB
    Loading...
  5. 2026-11-21 2-bit experts GGUF for one RTX 3090, model card, license 0305235 5.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration