base_model: PocketAiHub/Qwen3.8-27B-Abliterated-MLX
base_model_relation: quantized
model_name: Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp
library_name: mlx
pipeline_tag: image-text-to-text
license: apache-2.0
tags:

• mlx
• omlx
• apple-silicon
• 6-bit
• fp16
• mtp
• speculative-decoding
• qwen
• qwen3.8
• vision
• abliterated
• uncensored
• derivative

Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp

Unofficial community conversion. This repository is not affiliated with
or endorsed by Alibaba Cloud/Qwen, PocketAI Model Lab, or oMLX.

The 6bit/ variant of
PocketAiHub/Qwen3.8-27B-Abliterated-MLX, repackaged for oMLX on Apple Silicon:

1. FP16 instead of BF16. Every non-quantized tensor (norms, vision tower,
   quantization scales and biases) is stored as FP16. This targets oMLX's
   faster FP16 prefill path on M1/M2 Macs. The packed 6-bit weights are
   unchanged, so this is a storage-dtype conversion, not a requantization.
2. MTP head restored. The source config declares mtp_num_hidden_layers: 1,
   but its MTP tensors were stripped. This repository adds the unmodified
   Qwen3.8-27B MTP draft head (29 tensors, 6-bit group 64 / FP16) from
   metajazz/Qwen3.8-27B-oQ6-fp16-mtp at revision 2926178d6d49585ab826c961ba132ff75b34fa4e. They live in
   model-mtp.safetensors.

Content warning: the source model has been abliterated to suppress
refusals. It may produce harmful, illegal, offensive, or incorrect content
more readily than the upstream model. The upstream safety notice in
SOURCE_MODEL_CARD.md applies in full.

Why the grafted MTP head is compatible

PocketAI's abliteration (abliteration-manifest.json) projected a refusal
direction out of 80 residual-output matrices (attention and MLP output
projections) in layers 24–63. It did not touch the token embeddings or
lm_head, which the MTP head shares. (They are quantized slightly
differently: uniform 6-bit here, while the donor kept 8-bit embeddings.) The
head therefore sees slightly shifted final hidden states in the same
vocabulary and embedding space. The local
benchmark below shows it still predicts well: 92% draft acceptance on code and
reasoning.

Use with oMLX

hf download jongstra/Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp \
  --local-dir ~/.omlx/models/Qwen3.8-27B-Abliterated-MLX-6bit-fp16-mtp

-mtp models show no settings gear in oMLX's Model Manager. Add a
per-model entry to ~/.omlx/model_settings.json under models, then
restart oMLX. These are the settings used for the test below:

{
  "max_context_window": 150000,
  "temperature": 1.0, "top_p": 0.95, "top_k": 20, "min_p": 0.0,
  "presence_penalty": 0.0,
  "enable_thinking": true,
  "chat_template_kwargs": {"reasoning_effort": "medium"},
  "reasoning_parser": "qwen_3_5",
  "model_type_override": "vlm",
  "mtp_enabled": true,
  "mtp_adaptive_max_depth": 3,
  "specprefill_enabled": false,
  "dflash_enabled": false,
  "qwen35_ane_prefill_enabled": false,
  "turboquant_kv_enabled": false
}

oMLX 0.7.0 loads this checkpoint through its VLM engine and reports Lightning
MTP active. This repository contains MLX safetensors, not GGUF.

Local MTP benchmark

Mac Studio M2 Ultra (64 GB), oMLX 0.7.0. MTP on and off alternated in four
blocks. Each block ran every workload twice (temperature 1.0, top_p 0.95,
top_k 20). Figures are decode speeds.

With MTP off, the prose and reasoning means each include one slower outlier
run (19.7 and 22.5 tok/s). A cold 16K-token prompt took 55 s to the first
token, and one with 12,288 tokens cached took about 14 s. In a separate
reasoning sanity check the model gave the correct answer. These are local
observations from one machine, not a quality evaluation.

License and attribution

Qwen3.8-27B and both derivatives used here are Apache 2.0. LICENSE is the
source repository's Apache 2.0 text. The upstream card is kept verbatim in
SOURCE_MODEL_CARD.md, and PocketAI's abliteration-manifest.json,
artifact-manifest.json, and validation-summary.json are kept unchanged.
Note that the latter two describe PocketAI's BF16-retained files, not the
rewritten shards here. CHANGES.md lists every modification, and
conversion/fp16_graft.py is the exact script used.