← back to catalog · registered 2026-11-21 19:04

ghost-actual/Inkling-2-Mini-Uncensored-GGUF

ghost-actual GGUF MoE multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/ghost-actual%2FInkling-2-Mini-Uncensored-GGUF"
Response includes
  • classification m8
  • files 4
  • author_summary 4 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
0
Likes
0
Model age
today
created 2026-11-21
Downloads over time
Now 0 → from 0 ↑ 0%
0011 0 on Nov 21 0 on Nov 22 Nov
Nov 21 → Nov 22 · 2 snapshots · spans 1 day

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Quantizations
Q4_K
Tags
gguf llama.cpp inkling2 moe image-text-to-text uncensored abliterated base_model:OpenUncensored/Inkling-2-Mini-Uncensored base_model:quantized:OpenUncensored/Inkling-2-Mini-Uncensored license:apache-2.0 endpoints_compatible region:us

Related

Total size
16.3 GB
Files
4
Quantizations
3
Registered
2026-11-21 19:04
Last updated on HF
2026-11-21 18:24

Files by quantization

Q4_K 1 file 16.3 GB
Inkling-2-Mini-Uncensored-Q4_K_M.gguf 16.3 GB 3f34563f download
BF16 1 file 612 MB
mmproj-Inkling-2-Mini-Uncensored-BF16.gguf 612 MB 1ae9e4fe download
Auxiliary files 2 files 5.09 KB
README.md 3.46 KB 1f63ef5d download
.gitattributes 1.63 KB 9b199682 download

README current version from Hugging Face


license: apache-2.0
base_model:

  • OpenUncensored/Inkling-2-Mini-Uncensored
  • thinkingmachines/Inkling-2-Mini
    tags:
  • gguf
  • llama.cpp
  • inkling2
  • moe
  • image-text-to-text
  • uncensored
  • abliterated
    pipeline_tag: image-text-to-text

Inkling 2 Mini Uncensored — Q4_K_M GGUF

Q4_K_M quantization of OpenUncensored/Inkling-2-Mini-Uncensored
(Thinking Machines' Inkling 2 Mini, 26B total / 3.4B active, looped MoE, text + image input, 1M context,
with the refusals removed by directional ablation).

Built from the upstream safetensors (single shard, NVFP4 experts + MXFP8 + BF16 attention), dequantized to
BF16 by convert_hf_to_gguf.py, then quantized with mainline llama-quantize.

Files

File Size Notes
Inkling-2-Mini-Uncensored-Q4_K_M.gguf 17.48 GB Q4_K_M via mainline llama-quantize; 24 of 704 tensors fall back to default type (MoE expert tensors use q5_0/q4_K blocks)
mmproj-Inkling-2-Mini-Uncensored-BF16.gguf 0.64 GB Vision projector, converted from the same checkpoint (--mmproj)

The abliteration lives in the 20 attention output projections (attn.wo_ud), BF16 in the source. Those
tensors are quantized to q4_K along with the rest of the attention block, so the edit is carried at Q4
precision, not re-applied afterwards. A short behavioural check (a lock-picking request) returned a direct
technical answer with no refusal; no full refusal-rate or MMLU measurement was run on this quant.

How it was made

# mainline llama.cpp, after #32694 (inkling2_loop) and #32704 (NVFP4 scale fix)
hf download OpenUncensored/Inkling-2-Mini-Uncensored --local-dir Inkling2Mini

python convert_hf_to_gguf.py Inkling2Mini --outtype bf16 \
    --outfile inkling2-mini-uncensored-BF16.gguf

python convert_hf_to_gguf.py Inkling2Mini --mmproj --outtype bf16 \
    --outfile inkling2-mini-uncensored-mmproj-BF16.gguf

llama-quantize inkling2-mini-uncensored-BF16.gguf \
    inkling2-mini-uncensored-Q4_K_M.gguf Q4_K_M 20

Build of b1-19bc41d (master, 2026-11-21) with GGML_CUDA=ON.
Note: -compress-mode in ggml/src/ggml-cuda/CMakeLists.txt needs CUDA >= 12.9; on CUDA 12.8 the flag is
forwarded to the host compiler and breaks the build, so that guard was lowered locally.

Run

llama-server -m Inkling-2-Mini-Uncensored-Q4_K_M.gguf \
    --mmproj mmproj-Inkling-2-Mini-Uncensored-BF16.gguf \
    --jinja -ngl 99 -c 32768 --port 8001 \
    --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0

Recommended sampling from the upstream card: temperature 1.0, no truncation.
It is a thinking model by default — reasoning_content comes back separately; set
reasoning_effort: "none" (or enable_thinking: false in the chat template) for short answers.

Mainline llama.cpp does not yet support the DSpark speculative drafter for this model
(PR #32218 predates the inkling2_loop layout), so this repo has no drafter file.

Verification

  • Model loads and generates coherent output with mainline llama.cpp (b1-19bc41d), CPU-only smoke test:
    ~8.4 tok/s decode on a 24-thread host, no gibberish (the NVFP4 scale bug that broke earlier mainline
    builds of these weights is fixed in this build).
  • No MMLU / refusal / KL-divergence measurement was run on this quant. KLD against the unsloth UD-Q8_K_XL
    reference is not measured here.

Made by quantizing on release day; credit for the model to Thinking Machines and OpenUncensored.
Apache 2.0, inherited from the base model.

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-21 Upload folder using huggingface_hub 0bda4a2 3.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration