base_model: OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated
library_name: gguf
pipeline_tag: text-generation
tags:

• gguf
• llama-cpp
• imatrix
• quantized
• spark
• spark-x2.6
• heretic
• abliterated
• uncensored
• decensored
• reasoning
• roleplay
• edge-ai
• q4_k_m
• q5_k_m
• q6_k
• q8_0
• f16

Spark-X2.6-1.7B-Abliterated-GGUF (iMatrix Quantizations)

This repository provides Importance Matrix (imatrix) calibrated GGUF quantizations of OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated (an uncensored variant of XHToken/Spark-X2.6-1.7B abliterated with Heretic v2.0.0.dev0).

All quantized weights were calibrated using a 2,000-sample multi-domain dataset (spark_2000_mixed_dataset.jsonl) formatted strictly in the native Spark-X2.6 ChatML template (covering uncensored instruction following, coding, mathematics, science, and roleplay) to maximize perplexity retention and reasoning quality at low bit-widths.

Available GGUF Files

Prompt Format (Spark-X2.6 ChatML)

Spark-X2.6-1.7B uses a custom prompt structure with configurable thinking effort (low or high):

<｜start▁of▁sentence｜><|System|>
you are a helpful assistant.
`thinking_effort` guides on how much to think in your thinking channel (not including the response channel), supported values include `low` and `high`.
Now the system is invoked with `thinking_effort=high`.<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|User|>{prompt}<｜end▁of▁sentence｜><｜start▁of▁sentence｜><|Bot|><think>

How to Use

1. Using llama-cli (llama.cpp)

./llama-cli \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --temp 0.3 \
  --top-p 0.9 \
  --top-k 50 \
  --repeat-penalty 1.1 \
  -cnv

2. Using llama-server (OpenAI-Compatible Local API)

./llama-server \
  -m Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf \
  -ngl 99 \
  -c 4096 \
  --port 8080

3. Using llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="OpenIntelligenceNet/Spark-X2.6-1.7B-Abliterated-GGUF",
    filename="Spark-X2.6-1.7B-Abliterated-Q4_K_M-imatrix.gguf",
    n_gpu_layers=-1,
    n_ctx=4096
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Explain the difference between stack memory and heap memory in 3 concise bullet points."}
    ],
    temperature=0.3,
    top_p=0.9,
    top_k=50,
    max_tokens=512
)

print(response["choices"][0]["message"]["content"])