base_model: Mondk/Spark-X2.5-4B-Abliterated-Uncensored
license: other
library_name: mlx
pipeline_tag: text-generation
tags:

• mlx
• omlx
• spark2_5
• oq4e
• 4-bit
• quantization
• apple-silicon

Spark-X2.5-4B-Abliterated-Uncensored — MLX oQ4e (fp16 source)

Model Details

Model Description

oQ4e-quantized build of Mondk/Spark-X2.5-4B-Abliterated-Uncensored for
first-class, all-local serving on Apple Silicon via omlx (the MLX OpenAI-compatible server) and mlx-lm.

This is not a new model — it is a quantization of the existing Spark-X2.5 4B checkpoint, produced with
omlx's quantize_oq_streaming(..., oq_level=4, enhanced=True) pipeline:

• Why we made it: run a full 4B Spark-X2.5-class model locally in ~2.5 GB of memory at usable decode rates
  (≈44 tok/s on an M3 Pro), with the attention gates and projection heads kept at higher precision so reasoning
  quality survives 4-bit compression. The -fp16 suffix records that the quantizer consumed full-precision
  float16 source weights (best-fidelity input), as opposed to bf16 or already-quantized sources.

• Quantization: method oQ4e (omlx enhanced streaming imatrix calibration, 128×512 blocks), mixed-bit
  per-layer layout embedded in config.json (quantization / quantization_config) so the exact layout is
  restored automatically on load.

• Developed by: xunlinkx

• Shared by: xunlinkx

• Model type: spark2_5 text-generation decoder (MLX quantized, model_type: spark2_5)

• Language(s) (NLP): multilingual (base model; Spark-X2.5 pretraining, including English/Chinese coverage)

• License: the Spark-X2.5 series is Apache 2.0; this is a derived quantization / adaption build of the gated Mondk abliterated/uncensored checkpoint — see Mondk/Spark-X2.5-4B-Abliterated-Uncensored for those terms.

• Finetuned from model: Spark-X2.5-4B (base), abliterated/uncensored adaptation by Mondk; no additional fine-tuning performed here.

Model Sources

• Repository (quantization tooling): omlx — quantize/serve/stream via quantize_oq_streaming
• Base model: Mondk/Spark-X2.5-4B-Abliterated-Uncensored

Quantization Details

Per-layer overrides are written into config.json, so mlx-lm/omlx reproduce this exact layout at load time — no
manual flags needed. Calibration report: oq_imatrix_report.json (included in this repo).

Uses

Direct Use

Local and private inference on macOS (Apple Silicon):

# omlx — OpenAI-compatible server with native spark2_5 support
omlx serve -m xunlinkx/Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16

# or plain mlx-lm
python -m mlx_lm.generate \
  --model xunlinkx/Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16 \
  -p "Explain why attention gates matter under extreme quantization." \
  -m 256 --temp 1.0 --top-p 0.95  # official Spark-X2.5 sampling: temp 1.0, top_p 0.95, top_k -1

Requires an mlx-lm with native spark2_5 model support (module name matches the checkpoint's model_type);
omlx's pin provides this once the upstream model lands.

Recommended Sampling Settings

The official Spark-X2.5 card (XHToken/Spark-X2.5-4B) recommends:

• temperature = 1.0
• top_p = 0.95
• top_k = -1 (disabled)

These match the checkpoint's own generation_config.json (do_sample: true, penalties 0). All official
benchmarks were evaluated in thinking mode (enabled by default by the chat template). Use temperature 0
only for deterministic arithmetic/code checks (as in the upstream MLX quickstart).

Long-context (windowed attention, sliding window 512, RoPE base 5e6 for full-attention layers) works up to
max_position_embeddings = 1,048,576.

Out-of-Scope Use

• Do not fine-tune or continue-training from this quantized checkpoint — it is for inference only.
• Not a replacement for full-fidelity fp16/bf16 serving when maximum math/code accuracy is required (4-bit
  quantization is lossy; gates/projections are retained at 6–8 bit to bound the loss).
• This is an abliterated/uncensored checkpoint: it does not refuse or moderate. Do not deploy where
  unmoderated model output is unacceptable.

Bias, Risks, and Limitations

• Unmoderated output: the base model was abliterated to remove refusals and uncensored. Users get
  exactly what the weights produce — including content the upstream model would normally refuse. Deploy
  with guardrails or user-facing disclaimers where appropriate.
• Quantization loss: 4-bit group-64 affine quantization plus 6-bit projections can degrade
  formatting, long arithmetic, and rare-token recall versus fp16. The 8-bit g_proj and 6-bit
  projections are a deliberate mitigation.
• Windowed attention: sliding-window (512) layers mean very long prompts rely on repeating, rolling
  context; cross-window dependencies are best-effort.
• Verify outputs in high-stakes use; the model has no built-in safety tooling.

Recommendations

Users (both direct and downstream) should be aware of the above limitations, especially the uncensored
nature of the adaptation. Provide content warnings in any public-facing wrapper.

How to Get Started with the Model

See Uses → Direct Use. A minimal client example (OpenAI-compatible via omlx):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="omlx")
resp = client.chat.completions.create(
    model="Spark-X2.5-4B-Abliterated-Uncensored-MLX-oQ4e-fp16",
    messages=[{"role": "user", "content": "Write a haiku about quantization."}],
    temperature=1.0, top_p=0.95, max_tokens=128,
)
print(resp.choices[0].message.content)

Training Details

Training Data

No training data used here — this is a quantization artifact. See the base model card for the pretraining and
abliteration/uncensoring data details.

Training Procedure

Not trained. Produced by omlx quantize_oq_streaming, oQ4e mode, fp16 source, imatrix calibration at 128×512
streaming blocks.

Evaluation

Measured on Apple M3 Pro (36 GB):

• Load: strict single-file load ≈ 0.5 s
• Prefill: ≈ 145 tok/s (long prompts crossing the 512 sliding window handled correctly)
• Decode: ≈ 44 tok/s (200-token steady state), peak memory 2.5 GB
• Fidelity: 100 % argmax agreement with the fp32 reference over a 1,170-token prefill; cached-decode parity
  across the sliding-window boundary.

Technical Specifications

Model Architecture and Objective

Spark-X2.5 4B decoder (from config.json): 36 layers (mixed sliding-window/full attention), hidden size 2560,
intermediate 10240, 16→4 grouped-query attention, head dim 256, vocab 131072, sliding window 512, max position
1048576. RoPE: layer-type-specific partial dims/bases (SWA base 1e4 / full base 5e6). Head-wise sigmoid
attention output gating (the layer kept at 8-bit). Tied embeddings unprojected for MLX weight naming.

Compute Infrastructure

Hardware

Apple Silicon (developed on Apple M3 Pro, 36 GB). No cloud compute, no remote training.

Software

omlx (quantize + serve), mlx-lm (native spark2_5 module), MLX, Python 3.x.

Environmental Impact

Quantization and evaluation ran entirely on local Apple Silicon hardware; no cloud instances or dedicated
compute were used. Estimated carbon contribution of this artifact is negligible (single workstation, hours).

Model Card Contact

xunlinkx — questions/issues via the model repo discussion tab.