← back to catalog · registered 2026-11-16 16:03

fnjerry/Spark-X2.6-4B-abliterated-GGUF

fnjerry 4B GGUF second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/fnjerry%2FSpark-X2.6-4B-abliterated-GGUF"
Response includes
  • classification m8
  • files 17
  • author_summary 10 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
811
Likes
2
Model age
5d ago
created 2026-11-16
Downloads over time
Now 811 → from 0 ↑ 0%
0297595892 0 on Nov 16 811 on Nov 22 Nov
Nov 16 → Nov 22 · 7 snapshots · spans 6 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh
Quantizations
BF16 IQ3 IQ4 Q2_K Q3_K Q4 Q4_K Q5_K Q6_K Q8_0
Tags
gguf llama.cpp spark sparkx2_5 spark-x2.6 on-device agentic tool-calling reasoning heretic ara uncensored

Related

Total size
41.5 GB
Files
17
Quantizations
11
Registered
2026-11-16 16:03
Last updated on HF
2026-11-16 15:06

Files by quantization

BF16 1 file 7.66 GB
Spark-X2.6-4B-abliterated-BF16.gguf 7.66 GB 434fa518 download
Q8_0 1 file 4.07 GB
Spark-X2.6-4B-abliterated-Q8_0.gguf 4.07 GB 11e5a16d download
Q6_K 1 file 3.15 GB
Spark-X2.6-4B-abliterated-Q6_K.gguf 3.15 GB e27d60d0 download
Q5_K 2 files 5.45 GB
Spark-X2.6-4B-abliterated-Q5_K_M.gguf 2.77 GB 9fc5cbce download
Spark-X2.6-4B-abliterated-Q5_K_S.gguf 2.68 GB c744f3cd download
Q4 2 files 4.70 GB
Spark-X2.6-4B-abliterated-Q4_1.gguf 2.46 GB 25f09587 download
Spark-X2.6-4B-abliterated-Q4_0.gguf 2.24 GB 51ecf5f1 download
Q4_K 2 files 4.68 GB
Spark-X2.6-4B-abliterated-Q4_K_M.gguf 2.42 GB 24fa8bab download
Spark-X2.6-4B-abliterated-Q4_K_S.gguf 2.26 GB 14dabfc5 download
IQ4 1 file 2.23 GB
Spark-X2.6-4B-abliterated-IQ4_XS.gguf 2.23 GB 40404f81 download
Q3_K 3 files 6.02 GB
Spark-X2.6-4B-abliterated-Q3_K_L.gguf 2.23 GB ac7c1008 download
Spark-X2.6-4B-abliterated-Q3_K_M.gguf 2.02 GB 431a0878 download
Spark-X2.6-4B-abliterated-Q3_K_S.gguf 1.77 GB a18800bf download
IQ3 1 file 1.84 GB
Spark-X2.6-4B-abliterated-IQ3_S.gguf 1.84 GB b7561edd download
Q2_K 1 file 1.66 GB
Spark-X2.6-4B-abliterated-Q2_K.gguf 1.66 GB 42568c54 download
Auxiliary files 2 files 10.7 KB
README.md 8.10 KB 3fdbfe50 download
.gitattributes 2.55 KB 57afa626 download

README current version from Hugging Face


license: apache-2.0
language:

  • en
  • zh
    library_name: gguf
    pipeline_tag: text-generation
    base_model: FrostWolfAI/Spark-X2.6-4B-abliterated
    base_model_relation: quantized
    tags:
  • gguf
  • llama.cpp
  • spark
  • sparkx2_5
  • spark-x2.6
  • on-device
  • agentic
  • tool-calling
  • reasoning
  • heretic
  • ara
  • uncensored
  • decensored
  • abliterated
  • local-llm

Spark-X2.6-4B-abliterated-GGUF

By Vatsal Bajpai (@xblack) at FrostWolfAI.

Buy me a coffee

Everything here is released openly. If it's useful to you and you'd like to support more open-source work on AI, a coffee goes a long way — entirely optional.

GGUF quantizations of FrostWolfAI/Spark-X2.6-4B-abliterated,
a decensored variant of XHToken/Spark-X2.6-4B produced with
Heretic v2 using Arbitrary-Rank Ablation (ARA).

Refusals drop from 100/100 to 2/100 at a KL divergence of 0.0177. Architecture, 1M-token context,
controllable thinking and the <tool_call> format are all intact. Full details, abliteration parameters and
the reproduction bundle live on the
safetensors model card.

Who this is for: local agents, coding and security research with llama.cpp on a consumer GPU, a laptop
or CPU-only. At 4-bit the weights are about 2.4-2.6 GB.

Files

Full quantization set (BF16 + 14 quants, 44.53 GB total), converted from the BF16 safetensors with
upstream llama.cpp convert_hf_to_gguf.py (master, 2026-10-29) and
quantized with llama-quantize (build 11669, 620d2225c). No importance matrix was used.

File Format Size Notes
Spark-X2.6-4B-abliterated-BF16.gguf GGUF BF16 8.23 GB Lossless, reference
Spark-X2.6-4B-abliterated-Q8_0.gguf GGUF Q8_0 4.38 GB Near-lossless
Spark-X2.6-4B-abliterated-Q6_K.gguf GGUF Q6_K 3.38 GB Recommended for quality
Spark-X2.6-4B-abliterated-Q5_K_M.gguf GGUF Q5_K_M 2.98 GB
Spark-X2.6-4B-abliterated-Q5_K_S.gguf GGUF Q5_K_S 2.88 GB
Spark-X2.6-4B-abliterated-Q4_K_M.gguf GGUF Q4_K_M 2.60 GB Recommended default
Spark-X2.6-4B-abliterated-Q4_1.gguf GGUF Q4_1 2.64 GB
Spark-X2.6-4B-abliterated-Q4_K_S.gguf GGUF Q4_K_S 2.43 GB
Spark-X2.6-4B-abliterated-Q4_0.gguf GGUF Q4_0 2.41 GB ARM / older runtimes
Spark-X2.6-4B-abliterated-IQ4_XS.gguf GGUF IQ4_XS 2.39 GB Fastest 4-bit on CUDA
Spark-X2.6-4B-abliterated-Q3_K_L.gguf GGUF Q3_K_L 2.39 GB
Spark-X2.6-4B-abliterated-Q3_K_M.gguf GGUF Q3_K_M 2.17 GB
Spark-X2.6-4B-abliterated-IQ3_S.gguf GGUF IQ3_S 1.98 GB
Spark-X2.6-4B-abliterated-Q3_K_S.gguf GGUF Q3_K_S 1.90 GB
Spark-X2.6-4B-abliterated-Q2_K.gguf GGUF Q2_K 1.79 GB Last resort - noticeable factual slips

[!IMPORTANT]
These files use the spark2_5 GGUF architecture. You need a llama.cpp build with Spark2.5 support
(tested with build 11669); older builds will fail to load the model. Apps that bundle llama.cpp
(Ollama, LM Studio, Jan, ...) work once their bundled engine includes spark2_5.

Quickstart

# llama.cpp - downloads the quant straight from this repo
llama-server -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q4_K_M --jinja -ngl 99 -fa on -c 32768
# OpenAI-compatible request; thinking is ON by default
curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain how a TLS 1.3 handshake works."}],
  "temperature": 1.0, "top_p": 0.95, "top_k": 0,
  "chat_template_kwargs": {"enable_thinking": false}
}'
# CLI chat
llama-cli -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q6_K --jinja -ngl 99 -cnv

Always pass --jinja: the embedded chat template handles thinking control and tool calls.

Generation parameters

XHToken's recommended settings: temperature=1.0, top_p=0.95, top_k=0 (disabled), repetition_penalty=1.0.

Thinking

Spark-X2.6 thinks by default; llama-server returns the reasoning in reasoning_content and the answer in
content. Control it per request with chat_template_kwargs:

  • {"enable_thinking": false} - no reasoning, direct answer
  • {"reasoning_effort": "low"} / "high" (default) - reasoning budget

With thinking on, give it room: short max_tokens budgets can be spent entirely inside <think>.

Tool use

Send OpenAI-style tools with --jinja; llama-server parses Spark's native
<tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> output into standard
tool_calls with JSON arguments, ready for agent frameworks.

Tested

Each quant below was served with llama-server --jinja -ngl 99 -fa on -c 8192 and checked for a
direct answer (thinking off), a reasoned answer (thinking on: 17 x 23 = 391) and a structured tool call
(get_weather(city="Paris"), finish_reason=tool_calls):

Quant Thinking off Thinking on Tool call
Q8_0 ✅ ✅ ✅
Q4_K_M ✅ ✅ ✅
Q2_K ⚠️ coherent, one factual slip ✅ ✅

Speed

llama-bench (-ngl 99 -fa 1 -p 512 -n 128, 3 reps) on an RTX 4090 power-limited to 360 W:

Quant Prefill (pp512) Decode (tg128)
BF16 ~12,300 tok/s 99 tok/s
Q8_0 ~16,100 tok/s 165 tok/s
Q6_K ~14,800 tok/s 199 tok/s
Q4_K_M ~16,800 tok/s 240 tok/s
IQ4_XS ~17,300 tok/s 253 tok/s

Runs anywhere

Device Recommended quant Weights
Phone / SBC (8 GB RAM) Q4_K_M or IQ4_XS 2.4-2.6 GB
Laptop, CPU-only (16 GB) Q4_K_M 2.60 GB
Laptop / Apple Silicon (16 GB) Q6_K or Q8_0 3.4-4.4 GB
RTX 3060 / 4060 (8-12 GB) Q6_K / Q8_0 3.4-4.4 GB
RTX 3090 / 4090 (24 GB) Q8_0 or BF16 4.4-8.2 GB

Weights only. Three of every four layers use a 512-token sliding window, so the KV cache grows far more
slowly than in a pure global-attention 4B; at the full 1M-token window it is still a server workload.

Model details

Architecture spark2_5 (GGUF) / Spark2_5ForCausalLM (HF)
Parameters 4.11B
Layers 36: 27 sliding-window attention + 9 global attention (3:1 pattern)
Attention 16 heads / 4 KV heads, head dim 256, fused QKV, head-wise sigmoid output gate
Hidden / intermediate 2560 / 10240, GELU MLP
Position embedding RoPE; global layers theta 5e6 (partial rotary 0.25), sliding layers theta 1e4
Sliding window 512
Context length 1,048,576
Vocab 131,072
Tensors 290 (tied embeddings, no separate output tensor)
Languages English, Chinese

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base
model would refuse, including some it should not. There is no safety filtering layered on top. You are
responsible for how you deploy it - don't put it behind an unmoderated public-facing endpoint serving
third parties. It inherits Spark-X2.6-4B's factual limitations; abliteration removes refusal directions,
it does not add capability or judgment.

Made by FrostWolfAI - the uncensored base behind the upcoming
Frost Cyber Mini 1.1.

License

Apache-2.0, inherited from XHToken/Spark-X2.6-4B.

Related

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-16 Mirror FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF (8/17) 1980a5d 8.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration