language:
- en
- zh
license: apache-2.0
library_name: gguf
tags: - text-generation
- gsq
- rco
- llama.cpp
- quantized
- reasoning
- uncensored
- spark2_5
- spark-x2.6
- mixed-precision
base_model: XHToken/Spark-X2.6-4B
base_model_relation: quantized
pipeline_tag: text-generation
Spark-X2.6-4B-Uncensored GSQ-RCO GGUF
Mixed-precision GSQ-RCO GGUF quantizations of XHToken/Spark-X2.6-4B, optimized for efficient, high-fidelity local inference in llama.cpp.
Benchmark
Why GSQ-RCO?
Standard quantization severely degrades hybrid-attention architectures (e.g. stock low-bit quantizations cause representation drift and degradation in reasoning coherence).
GSQ-RCO solves this through two targeted techniques:
- RCO: Keeps all 36 sigmoid headwise attention gates (
g_proj) and the 9 global full-attention projections in unquantized BF16. - GSQ: Optimizes per-layer precision guided by Hessian sensitivity calibration.
GGUF Files
| File | bpw | Size |
|---|---|---|
| Spark-X2.6-4B-GSQ-RCO-IQ2_M.gguf | 2.60 | 1.64 GB |
| Spark-X2.6-4B-GSQ-RCO-IQ3_S.gguf | 3.25 | 2.05 GB |
| Spark-X2.6-4B-GSQ-RCO-IQ3_M.gguf (Recommended) | 3.75 | 2.28 GB |
| Spark-X2.6-4B-GSQ-RCO-IQ4_XS.gguf | 4.25 | 2.59 GB |
Usage
llama.cpp
Interactive Chat
llama-cli \
-m Spark-X2.6-4B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 16384 \
-cnv
Local Server (OpenAI-Compatible)
llama-server \
-m Spark-X2.6-4B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 32768 \
--port 8080
128k Long-Context (Low-VRAM)
Using 4-bit KV cache (-ctk q4_0 -ctv q4_0) allows 131k context to run within ~3.5 – 4 GB total VRAM:
llama-server \
-m Spark-X2.6-4B-GSQ-RCO-IQ3_M.gguf \
-ngl 99 \
-fa \
-b 2048 \
-c 131072 \
-ctk q4_0 -ctv q4_0 \
--port 8080
Ollama
ollama run hf.co/abenzerps/Spark-X2.6-4B-Uncensored-GSQ-RCO-GGUF:IQ3_M
Attribution & License
- Distributed under the Apache-2.0 License, matching upstream XHToken/Spark-X2.6-4B.
- Base model: XHToken/Spark-X2.6-4B.