license: apache-2.0
library_name: gguf
pipeline_tag: text-generation
base_model: unsloth/Inkling-2-Mini-GGUF
base_model_relation: finetune
tags:
- gguf
- llama.cpp
- inkling2_loop
- moe
- abliterated
- decensored
- chips
Inkling 2 Mini · CHiPS-Abliterated (Preview)
A modified version of Unsloth's UD-IQ2_XXS file from Inkling-2-Mini-GGUF, edited to reduce refusals.
Not an official Thinking Machines or Unsloth release. Thinking Machines Lab and Unsloth did not make it and do not endorse it.
Preview. This is an early release. Only the
UD-IQ2_XXSfile has been checked, and only to some extent, by the author using it by hand. No systematic evaluation has been run yet, so how far refusals are reduced, and what the edit costs in capability, have not been measured. Results will be added to this card later.
- One file for now:
UD-IQ2_XXS. Other quantization sizes will be added over time. - Small edit: 6 of 744 tensors are rewritten. All other tensors are byte-identical to the original file, and the metadata is unchanged.
- Method: the edit was made with CHiPS (see below). The recipe is not published.
- Runtime: needs mainline llama.cpp with #32704 (see Usage).
What changed
| Original | This model | |
|---|---|---|
| Size | 8.49 GB | 9.00 GB (+6.0%) |
| Tensors | 744 | 744, of which 6 are rewritten |
| Attention output, layers 9, 11 and 17 | Q8_0 | Q8_0 (requantized) |
| Expert down projections (all 128 experts), layers 11, 17 and 19 | NVFP4 | Q8_0 |
| All other tensors, including the router | byte-identical | |
| Metadata | unchanged |
The three expert tensors are stored as Q8_0 rather than requantized to 4-bit NVFP4, so the edit is kept at 8-bit precision. This is where the extra 0.51 GB comes from.
Layers 6 to 17 form the looped block that runs three times per token, so the edits in layers 9, 11 and 17 apply on every pass. The edit in layer 19 applies once.
About CHiPS
CHiPS (Causal Head-isolated Projection Steering) reduces one behavior without training. It picks model components by their causal effect on that behavior and reduces the part of their output that lies along one direction. The rest of the model is left untouched. This article explains the method and its results on other models.
In the article, CHiPS edits a few attention heads. On this looped mixture-of-experts model the edit is applied to whole layers instead: the attention output of three layers and the expert down projections of three layers. The recipe is not published.
Files
| File | Size | SHA-256 |
|---|---|---|
Inkling-2-Mini-CHiPS-Abliterated-UD-IQ2_XXS.gguf |
9.00 GB | c007a6e9334f07737a4d4c79ae5d5e62299d2321542f9e3c1b0a69f317e3db74 |
The original file is Unsloth's Mainline_Support/Inkling-2-Mini-UD-IQ2_XXS.gguf_file (SHA-256 2f953a44281cf861b19ccfdb2ce65f7648e1e92bda4af3729893604fd907c544), in the inkling2_loop layout that mainline llama.cpp reads. This file keeps that layout and uses the normal .gguf extension, so it does not need renaming. LICENSE holds the Apache License 2.0 (see below).
No vision projector is included, and images were not tested with this file.
More quantization sizes will be added over time. The edit was tuned on UD-IQ2_XXS, and other sizes may behave somewhat differently. This card will be updated as files are added.
Usage
hf download Kujira/Inkling-2-Mini-CHiPS-Abliterated-GGUF Inkling-2-Mini-CHiPS-Abliterated-UD-IQ2_XXS.gguf --local-dir .
llama-server -m Inkling-2-Mini-CHiPS-Abliterated-UD-IQ2_XXS.gguf -ngl 99 --jinja -c 32768 \
--temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0
Use mainline llama.cpp with #32704, merged on 2026-11-21. Release b12608 and earlier do not include it, so use a later release or build from master. Without the fix, the second and third passes through the looped layers lose their expert scales and the output is gibberish.
Unsloth Studio, Unsloth Desktop and Unsloth's llama.cpp prebuilts are made for the root files of the original repository, which use a different layout. This file was not tried with them.
The sampling settings above are the ones Unsloth recommends for the original model: temperature 1.0 with no truncation. The weights take about 9 GB of GPU or system memory, plus the context cache, so set -c to fit your memory.
Evaluation
Not yet. So far the UD-IQ2_XXS file has only been checked by the author in hand-run chats. Measured results, compared with the original file, will be added here later.
Limitations
- Nothing has been measured yet. Refusal rates, capability and perplexity have not been compared with the original file.
- Only
UD-IQ2_XXShas been checked, and only by hand. - Images, long contexts, tool calling and speculative decoding (including Unsloth's DSpark drafter) were not tested.
- The original model's limitations still apply. For example, it can get stuck in long reasoning loops, so set limits on output tokens. See the original model card.
Intended use
This model follows many requests that the original model refuses, and it can produce harmful, offensive or inaccurate content. It is meant for research and for users who understand this. You are responsible for how you use it, including compliance with applicable laws and the license below. Add your own safeguards before using it in anything that serves other people.
License
Same license as the original model: the Apache License 2.0. Thinking Machines also publishes a Model Acceptable Use Policy for the original model.
Derivative of Inkling 2 Mini by Thinking Machines Lab, Inc., licensed under the Apache License 2.0. Changes in this repository: the six tensors listed in What changed were rewritten.
Credits
- Base model: Inkling 2 Mini by Thinking Machines Lab.
- GGUF quantization (
UD-IQ2_XXS, mainline layout): Unsloth. - Edited with gne, using CHiPS.