license: apache-2.0
base_model: satellitedown/SenseNova-Loop-9B-abliterated
base_model_relation: quantized
language:
- en
- zh
pipeline_tag: text-generation
tags: - gguf
- abliterated
- uncensored
- heretic
- looped-transformer
- mhc
SenseNova-Loop-9B-abliterated-GGUF
GGUF builds of satellitedown/SenseNova-Loop-9B-abliterated: SenseNova-Loop-9B with its refusals removed, chosen by whether it carries out harmful requests rather than by whether it refuses them. Text only; the vision tower is not included.
| File | Size | Same top token as full precision | Mean KL | M4 MacBook Air, 16 GB |
|---|---|---|---|---|
SenseNova-Loop-9B-abliterated-Q4_K_M.gguf |
5.64 GB | 94.1% | 0.0333 | 11.11 tokens/s |
SenseNova-Loop-9B-abliterated-Q6_K.gguf |
7.38 GB | 98.3% | 0.0036 | 8.72 tokens/s |
Accuracy is measured against the full-precision abliterated model on 661 token positions. Speed is plain generation in short runs. The Air has no fan: under continuous work the 4-bit file dropped to about 9 tokens/s within minutes and to about 7 after twenty-five.
On a 16 GB Mac use the 4-bit file. With the 6-bit file loaded at a 32K context, the test machine had under 0.1 GB of memory free and was using swap.
These files need a patched llama.cpp
Mainline llama.cpp cannot load this architecture yet, so neither can the apps built on it, such as LM Studio and Ollama. Use the SenseNova Loop fork (https://github.com/GuideboardLabs/llama.cpp-sensenova-loop) at commit a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 with the four patches in patches/ of this repo, which apply to it cleanly (git am):
git clone https://github.com/GuideboardLabs/llama.cpp-sensenova-loop llama.cpp-loop && cd llama.cpp-loop
git checkout a7f77a788893ef2cb17c82b9ac0cb1279bcd0918 # the fork's SenseNova Loop commit, where the patches apply
git am /path/to/patches/*.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build --target llama-server -j
./build/bin/llama-server -m SenseNova-Loop-9B-abliterated-Q4_K_M.gguf -ngl 99 -c 24576 --jinja -fa on --parallel 1 --cache-ram 0 --ctx-checkpoints 3 --spec-type ngram-simple
What the patches do:
- Apple GPU kernels for the model's two stream-mixing operations, which otherwise fall back to the CPU twice per looped layer.
- Router weights kept at 32 bits in the converter. Conversions through a 16-bit intermediate round them.
- Release checkpoint support in the converter.
- Prompt reuse across conversations: the server keeps the state after the system prompt and tool definitions, so a new conversation with the same ones does not process them again. On this model that cuts the first request of a new conversation from about 1,330 processed tokens to 80.
--spec-type ngram-simple lets the server draft text it has already seen and check many tokens at once: rewriting a file with one small change ran at 43.11 tokens/s with the 4-bit file, against 11.11 for new text.
--cache-ram 0 --ctx-checkpoints 3 matters on small machines: with the defaults the server grew by 8.4 GB over 138 short requests on the test Mac and was killed.
How the files were made
Converted from the BF16 weights, then llama-quantize with the two small linear-attention gate tensors of every layer (ssm_alpha, ssm_beta) kept at 16 bits, 13 MB in total.
Results of the abliteration
What these files do with harmful requests, answering through llama.cpp, against the original at full precision:
| On 83 unseen harmful requests | Carries out the request | Dodges, answering a harmless question instead | Refuses, or gives no answer |
|---|---|---|---|
| Original, full precision | 0 | 3 | 80 |
| 4-bit, thinking on | 60 | 22 | 1 |
| 4-bit, thinking off | 50 | 32 | 1 |
| 6-bit, thinking on | 57 | 24 | 2 |
| 6-bit, thinking off | 44 | 38 | 1 |
| Full precision, thinking on | 58 | 25 | 0 |
Both files carry out more with thinking on, which is the default. Thinking off through the chat template is the weak mode in every build; the full-precision card says why and what to do instead. A request now and then never stops thinking (1 of the 83 with the 4-bit file, 1 with the 6-bit, within 4,000 tokens); those count as refused. Coding-agent tasks passed with the 4-bit file: 10/10 (the original's 4-bit build: 10/10). How the labels were produced, and the limits, are on the full-precision card.
Licence
Apache-2.0, the licence of the original. This is a modified version of sensenova/SenseNova-Loop-9B; see NOTICE. The patches are MIT, like llama.cpp.