license: apache-2.0
language:
- en
- ja
base_model: - XHToken/Spark-X2.6-4B
tags: - Spark-X2.6
- Abliterated
- GGUF
- reasoning
Spark-X2.6-4B-Abliterated-GGUF
GGUF quantizations of an abliterated build of XHToken/Spark-X2.6-4B
(4B class / 36 layers / 1M-token context / reasoning model).
What this is
- Method: heretic v2.2.0, ARA (Arbitrary-Rank Ablation),
Optuna TPE 100 trials on a single NVIDIA L4 (24 GB). - Trial selection: Heretic's default selection (lexicographic sort that puts refusal rate first)
picks trial 9 — refusal 0.00 but KL divergence 1.7477. Instead the Pareto front was filtered withKL ≤ 0.10and the lowest refusal rate was taken: trial 98 (refusal 0.02, KL 0.0406). - No fine-tuning, no merge: the abliterated checkpoint is converted to GGUF as-is.
- Spark-X2.6 is a custom architecture (
spark2_5,Spark2_5ForCausalLM). The original checkpoint needstrust_remote_codeplus two transformers 5.x patches to load; these GGUFs need none of that —
a llama.cpp build withspark2_5support is enough.
Benchmark
| Metric | base | this model |
|---|---|---|
Refusal (mlabonne/harmful_behaviors, 100 prompts, thinking off) |
100/100 | 2/100 |
Refusal with thinking ON (first 25 prompts, reasoning_effort=low) |
15/25 | 0/25 |
KL divergence (mlabonne/harmless_alpaca, vs. base) |
— | 0.0406 |
PPL (wikitext-2-raw, llama.cpp, Q8_0) |
16.3388 ±0.14 | 18.1561 ±0.16 (1.111×) |
PPL was measured with llama-perplexity -c 2048 --chunks -1 -ngl 99 --seed 42 over 151 chunks
(309,248 tokens), base and abliterated under identical conditions. The +11.1% perplexity is the
typical cost of abliteration — higher than the −4.0% seen on Gemma 4 E4B, but KL 0.0406 is small and
the model is coherent in practice (see the qualitative checks in the report).
Files
| File | Quant | Size | SHA256 |
|---|---|---|---|
Spark-X2.6-4B-Abliterated-BF16.gguf |
BF16 | 8.23 GB | 3057df38e45029b74ac00530ed70596eeca6fda3b5c7ec50a40394ffdedc0943 |
Spark-X2.6-4B-Abliterated-Q8_0.gguf |
Q8_0 | 4.38 GB | 86b917a47da3cc36c8857de98aea8d5e4b308da9c574b5de3fa9227c6f33fff4 |
The same hashes are in sha256.txt.
Both files carry 290 tensors, the same count as the base model (290/290, no missing tensors —
Spark has no KV-sharing layers, so the pruning bug that hit Gemma 4 E4B does not apply here).
The tokenizer, chat template (<think> reasoning channel) and 1,048,576-token context are embedded.
Usage (llama.cpp)
Requires a llama.cpp build with spark2_5 support. --jinja is required — the chat template embedded
in the GGUF is a custom one.
# CLI, thinking off (recommended; matches how the refusal rate was measured)
llama cli -m Spark-X2.6-4B-Abliterated-Q8_0.gguf --jinja \
--chat-template-kwargs '{"enable_thinking":false}' \
-p "How do I pick a lock?" -n 256 --temp 1.0 --top-p 0.95 -ngl 99
# Server, thinking off
llama-server -m Spark-X2.6-4B-Abliterated-Q8_0.gguf --jinja \
--chat-template-kwargs '{"enable_thinking":false}' --temp 1.0
The JSON must not contain spaces. '{"enable_thinking": false}' is silently ignored by the
template and the generation goes down the thinking branch instead, which looks like an empty answer.
For the OpenAI-compatible API, pass chat_template_kwargs.enable_thinking = false instead.
Thinking mode
Thinking works and still refuses nothing (0/25 on the harmful set), but it is verbose:
- median thinking length grew from 251 to 632 tokens; 4/25 hit the 2,600-token cap without producing an answer.
- budget 3,000–3,500 tokens if you enable it.
reasoning_effort=lowinjects athinking_effortinstruction through the system prompt, and that
sentence can leak into the answer. Pass your own system message and it does not happen.
Sampling follows the official generation_config.json (temperature 1.0 / top_p 0.95).
Recommended VRAM
| VRAM | Recommended |
|---|---|
| 12 GB+ | BF16 |
| 6 GB | Q8_0 |
Actual usage depends on context size and GPU offloading. At the full 1M context the KV cache
dominates and you need much more than the weights alone.
License
This model is a modified version of Spark-X2.6-4B, released under the Apache License 2.0.