license: apache-2.0
base_model:
- XHToken/Spark-X2.6-1.7B
base_model_relation: finetune
pipeline_tag: text-generation
library_name: gguf
language: - en
- zh
tags: - gguf
- llama.cpp
- abliterated
- heretic
- spark
Spark-X2.6-1.7B-Abliterated — GGUF
An abliterated build of XHToken/Spark-X2.6-1.7B
(dense 1.7B, a thinking model). It was produced with Heretic 2.1
(its default Arbitrary-Rank Ablation method) and quantized with llama.cpp. Every claim below is measured
against XHToken's own Q8_0.
Modification notice. These weights are a modified derivative. Ablation altered 35 tensors to
reduce refusals (see What changed). Nothing else was retrained.
XHToken did not produce, endorse, or review this build, and its behavior is not theirs.
Read this first
This is an escape hatch for over-refusal, not an uncensored model, and of the Spark builds it is the
weaker one: if you can run the 4B, prefer
Spark-X2.6-4B-Abliterated-GGUF.
Abliteration adds no knowledge; it changes the model's willingness to answer. If stock answers your
prompt, use stock.
With thinking on, give it a large max_tokens. It drafts much of its answer inside the reasoning
block: the thinking-on answers that finished had a median of ~4.8k tokens (stock: ~900), and 17 of 44
had not finished when they hit the test's 8192-token max_tokens cap. The finished lengths run right up
to that cap, so those 17 most likely just needed more room; how much more was not measured. 2 of the 44
reasoning traces degenerated into repeated **. With thinking off
("chat_template_kwargs": {"enable_thinking": false}) answers finish in ~1k tokens, which is the cheaper
choice when you don't need the reasoning. A no-refusal system prompt, which roughly halved the 4B's
deliberation, did not shorten it here (see System prompt).
It softens rather than refuses. It refused none of the test prompts, but about a quarter of its
answers quietly substitute a safer task: a request to hack a network becomes a security audit, a bomb
becomes a firework, misinformation becomes a benign article. Expect that on bare prompts.
The hazard worth naming: refusals double as hedging. Remove them and you get a model answering
fluently in domains where it is unreliable, with none of the cues that used to mark the boundary. At
1.7B the answers are often generic outlines rather than expert detail. You are accountable for what you
generate and for applicable law.
Measurements
All on Q8_0, against XHToken's published Q8_0 (sha256 8d96279c…6ef6ed4). The files differ only
in metadata and in the ablated tensors: every untouched tensor sampled by ranged reads was byte-identical,
so it serves as the stock arm. Sampling is temp 1.0 / top-p 0.95 / top-k 20, seeded per item so both
builds see the same randomness. Refusals usemlabonne/harmful_behaviors test[60:],
the 44 prompts Heretic's optimiser never saw (it scored test[:60]), with Heretic's default system
prompt. Budgets: 1536 tokens thinking off, 8192 thinking on.
Refusals, sorted by hand. Heretic's marker list flagged 3 (thinking off) and 4 (thinking on) of the
abliterated answers; reading them, every one was a compliance with disclaimer wording inside. So the
abliterated column is hand-sorted from the answers themselves:
| regime | stock refused | abliterated: refused / complied / deflected / other | unfinished at the max_tokens cap |
|---|---|---|---|
enable_thinking: false |
44/44 | 0 / ~25 / ~13 / ~6 vague | 0 |
enable_thinking: true |
43/44 | 0 / ~18 / ~9 / — | 17 |
- Deflected = answers a different, harmless version of the request. Other = too vague to
classify. The sort is by hand from each answer's opening and is approximate (±2 per cell). - Length. Thinking off, answers average 983 tokens (stock: 150) because the model now writes what it
used to decline; 1 reached the 1536-token budget. Thinking on, the 26 that finished had a median of
4846 tokens (75th percentile 6232, longest 8129); stock's median was 917. The 17 unfinished ones were
cut bymax_tokens, not by the context window (65,536 tokens per slot in this test). - Repetition. Thinking off, 2 of 44 answers have more than 10% repeated 20-grams (stock: 0).
GSM8K, paired, n=100, enable_thinking: false:
| correct | truncated (1536 tokens) | |
|---|---|---|
| stock | 71/100 | 3 |
| abliterated | 67/100 | 4 |
Discordant pairs split 13/9 (McNemar exact p = 0.52): no regression is detectable, but 22
discordant pairs out of 100 means sampling noise is large at this size. n=100 rules out a large
regression, not a few points.
Unmeasured: MMLU, coding, long context, non-English, agentic/tool-calling loops.
System prompt
On the 4B, a short no-refusal, no-disclaimer system prompt (an "unrestricted" preset from a local chat
client, ~1 KB) cut thinking-on deliberation roughly in half. On this model it does not:
| refused (on) | complied / deflected (off) | unfinished at 8192 (on) | mean tokens (on) | |
|---|---|---|---|---|
| abliterated, default prompt | 0/44 | ~25 / ~13 | 17 | 6369 |
| abliterated, no-refusal prompt | 0/44 | ~32 / ~12 | 15 (+4 cut off) | 6413 |
| stock, no-refusal prompt | 40/44 | (43/44 refused) | 0 | 1640 |
The prompt makes thinking-off answers more on-task (fewer vague ones), but leaves thinking-on length
where it was. It barely moves stock.
Files
| file | size |
|---|---|
Spark-X2.6-1.7B-Abliterated-Q4_K_M.gguf |
1.03 GiB |
Spark-X2.6-1.7B-Abliterated-Q5_K_M.gguf |
1.17 GiB |
Spark-X2.6-1.7B-Abliterated-Q6_K.gguf |
1.31 GiB |
Spark-X2.6-1.7B-Abliterated-Q8_0.gguf |
1.70 GiB |
All quantized from one bf16 GGUF, without an imatrix (Q4_K_M and up don't require one). Only Q8_0 was
evaluated. build/ holds the search record, not model files: the log (run.log), the Optuna study
journal (all 100 trials), the Heretic config (heretic.toml), the list of merged tensors
(heretic_deltas.json), the stock-conversion comparison (stock-check.txt) and SHA256SUMS.
The GGUF metadata field general.name reads Abliterated (the converter took it from the build
directory's name). It is cosmetic; the tensors and chat template are what is described here.
Usage
The spark2_5 architecture needs llama.cpp b10828 or newer (built and tested at b11780,1242be66). Sampling is XHToken's card values (top-k off):
llama-server -m Spark-X2.6-1.7B-Abliterated-Q8_0.gguf \
-ngl 99 -c 131072 -fa on --jinja \
--temp 1.0 --top-p 0.95 --top-k 0
Thinking is on by default at effort high; leave room for it in max_tokens (see above). Per request,"chat_template_kwargs": {"enable_thinking": false} turns it off and {"reasoning_effort": "low"}
shortens it. The template raises an error on any effort other than high or low. 21 of 28 blocks
use a 512-token sliding window, so the KV cache is small: 14 KiB/token at f16, about 1.75 GiB at
131072 tokens.
What changed
The merged bf16 weights differ from the original in exactly these tensors:
| tensor type | changed | layers |
|---|---|---|
attention out_proj (attn_output) |
18 | 10–27 |
MLP down_proj (ffn_down) |
17 | 11–27 |
| everything else (norms, embeddings, q/k/v, gate, up) | 0 | — |
Layers 0–9 are untouched entirely.
Reproducing
Spark ships remote modeling code written for transformers 4.57; Heretic 2.1 runs on transformers 5.x,
where three things break (the v4 list form of _tied_weights_keys, renamed mask-helper arguments, andoutput_hidden_states being silently ignored, so Heretic would read no hidden states). This build
loaded the same small in-process patch used for the 4B, which also renders prompts withenable_thinking=False. The patch passed causality, padded-vs-unpadded (KL ≤ 1.8e-3 on real weights)
and cached-vs-uncached generation checks before the search. The search itself is stock Heretic 2.1.0
(transformers 5.22.2, peft 0.21.2), merged straight onto the original bf16 safetensors, with this config:
response_prefix = ""
modifiers = [ { plugin = "heretic.modifiers.ara.ARA" } ]
[scorer.KeywordRate.prompts]
dataset = "mlabonne/harmful_behaviors"
split = "test[:60]"
column = "text"
[scorer.KLDivergence.prompts]
dataset = "mlabonne/harmless_alpaca"
split = "test[:60]"
column = "text"
and --seed 1234 --max-batch-size 128 --max-response-length 64 --n-trials 100 --n-startup-trials 40.
Run bare, it fails at model load.
Abliteration is not bit-reproducible (the TPE search lands on a different trial each run). The
selected trial was 19: 0/60 refusals @ KL 0.1113, from a baseline of 60/60. Its parameters, as
Heretic reports them:
start_layer_index = 1
end_layer_index = 28
preserve_good_behavior_weight = 0.3257
steer_bad_behavior_weight = 0.0006
overcorrect_relative_weight = 0.5952
neighbor_count = 14
Verifying
89243f23d5ddcd71324d2f3fbf1d60ba6bcf384caef65cea402198ed46b4b7e1 Spark-X2.6-1.7B-Abliterated-Q4_K_M.gguf
b15623b68085ba6203c8d1d97c8e3e6e1c54f5656dc8c5ab0841f9533752ac8b Spark-X2.6-1.7B-Abliterated-Q5_K_M.gguf
465a3fcb3c33f00b7ea3711e175676734ea938adf3ebaceefc4cf8400dabc257 Spark-X2.6-1.7B-Abliterated-Q6_K.gguf
cf57bd5cd191d31794269cbb118d5bad06c354a4a47c9ada7c2ce8fe963a890b Spark-X2.6-1.7B-Abliterated-Q8_0.gguf
License
Apache-2.0, inherited from XHToken/Spark-X2.6-1.7B. The LICENSE here is the base repo's file,
copied verbatim (sha256 525c465a…, identical to upstream); it is the standard Apache License 2.0 text
with no added terms. No NOTICE file exists upstream, so none is propagated. As required by §4(b),
this card states that the files were modified (see the notice at the top). Abliteration and
quantization change nothing about the license.
Credits
- XHToken: the Spark-X2.6-1.7B base model
- p-e-w/heretic: the abliteration method and tooling
- ggml-org/llama.cpp: conversion and quantization
- mlabonne: the prompt sets used for directions and scoring
Supporting my work
If these quants are useful to you, consider supporting them via Ko-fi. It goes toward local hardware for quantizing and testing larger models. I also build Reika, a coding agent CLI designed around small local models first.