license: apache-2.0
base_model: insraq/MiniCPM5-2B-heretic-abliterated
library_name: mlx
tags:

• mlx
• mlx-lm
• minicpm
• minicpm5
• llama
• abliterated
• uncensored
• 8bit

MiniCPM5-2B-heretic-abliterated MLX 8-bit

MLX 8-bit quantization (8.5 bits/weight, group size 64) of
insraq/MiniCPM5-2B-heretic-abliterated,
a Heretic v1.4.0 abliteration (decensoring) of
openbmb/MiniCPM5-2B.
Runs on Apple Silicon via mlx-lm.
~2.5 GB on disk, ~2.8 GB peak memory, ~87 tok/s on an M4 Pro.

Converted from the fp16 safetensors (not the GGUF) so no double-quantization loss.

Important: this repo includes a rope config fix

The source repo's config.json was exported with transformers v5, which nests
the rope settings as rope_parameters: {rope_theta: ...}. mlx-lm's llama
loader only reads the flat rope_theta key and silently falls back to the
default (10000) when it is missing. The result: the model runs with a rope
theta 500x too small, which is coherent at short context and collapses into
word salad somewhere past ~12k tokens, regardless of quant, sampler, or
thinking mode.

This repo's config.json carries the flat "rope_theta": 5000000 key, so it
works correctly out of the box. With the fix, long context behaves as the base
model card claims: verified correct last-line retrieval from 70k-token
documents on a 2B model.

If you convert transformers-v5-era re-exports to MLX yourself, check for nested
rope_parameters and add the flat key before serving.

Usage

from mlx_lm import load, generate

model, tokenizer = load("bumblebuttpow/MiniCPM5-2B-heretic-abliterated-MLX-8bit")

Or serve it:

mlx_lm.server --model bumblebuttpow/MiniCPM5-2B-heretic-abliterated-MLX-8bit

Sampling recommendations (measured)

This merge has a sensitive think channel. Recommendations:

• temperature 0.6, top_k 20, repetition_penalty 1.05 — stable. Higher
  temperature (including the base model's recommended 1.0) causes the think
  channel to verbatim-loop on trivial prompts.
• Thinking mode: it deliberates at length. Give it max_tokens of 800+ or the
  answer stays trapped in the reasoning block.
• Instant-answer mode: pass
  chat_template_kwargs: {"enable_thinking": false} to pre-close the think
  block (supported by the chat template).

Verification

• 8-bit quant from fp16 safetensors via mlx_lm 0.31.3.
• Short-context QA, 26k and 70k-token retrieval prompts: coherent, correct
  retrieval, no degeneration (after the rope fix above).
• Verified both solo and pipeline-parallel across two Macs with identical
  outputs.

Attribution

• Abliteration: insraq (Heretic v1.4.0, KL 0.0391, refusals 5/100)
• Base model: openbmb/MiniCPM5-2B (Apache 2.0)
• Quant + rope fix: bumblebuttpow