license: apache-2.0
base_model: windowsxp811203/Qwen3.8-27B-Abliterated
pipeline_tag: image-text-to-text
library_name: mlx
tags:

• mlx
• omlx
• bf16
• mtp
• native-mtp
• apple-silicon
• qwen3_5
• qwen3.8
• qwen
• abliterated
• uncensored
• vision
  language:
• en
• zh

Qwen3.8-27B-Abliterated — MLX bf16 with native MTP

Unquantized MLX conversion of windowsxp811203/Qwen3.8-27B-Abliterated
(an abliterated Qwen/Qwen3.8-27B) with the MTP draft head and the vision
tower kept in the checkpoint. 51.75 GiB (55.56 GB) of safetensors — this is the reference the quantized builds were made from,
and the right starting point if you want to run your own oQ / mlx-vlm quantization with MTP preserved.

Quantized siblings (same conversion, oMLX oQ with preserve_mtp=True — MTP head kept, its Linears at the base bits, mtp.fc left bf16): oQ4e/ ((15.81 GiB)) · oQ6e/ ((22.09 GiB)) · oQ8e/ ((27.94 GiB)) — all in the project repo.

What is in the file

• 1199 tensors, all bf16: language model (64 layers, 48 GatedDeltaNet + 16 full attention), vision tower 333 tensors,
  MTP head 15 tensors (language_model.mtp.*, mtp_num_hidden_layers: 1), lm_head untied.
• MLX conventions applied by the converter: the zero-centered RMSNorm weights carry the +1 offset (input/post-attention
  layernorms, q_norm/k_norm, the final norm and every MTP norm including pre_fc_norm_*; the GDN linear_attn.norm
  gated RMSNorm is left as-is), the GDN conv1d is transposed to (channels, kernel, 1), and the vision
  patch_embed.proj.weight is moved to channels-last (out, T, H, W, in).
• Provenance: converted with mlx_vlm.convert(quantize=False, dtype=bfloat16) under oMLX 0.6.4's MTP sanitize
  patch (mlx-vlm 0.6.3 @ 78b96eb, mlx 0.32), straight from the HF bf16 parent with no dtype change (all 1199 tensors bf16); apart from the three MLX layout conventions above, tensors are copied unchanged.

Verification

• Probe against the HF parent: 15/15 MTP and 333/333 vision tensors present, all 1199 tensors bf16, and the sampled RMSNorm offsets (7 MTP + 3 trunk norms) are +1.000 within bf16 rounding.
• MMLU (oMLX accuracy bench: 5-shot, its seeded 400-question sample stratified by subject, temperature 0, non-thinking, MTP enabled at depth 3 — the same settings as the oQ runs; single-letter answers barely exercise the draft path, so the score is effectively independent of MTP): 82.50 % (330/400) — the reference the oQ cards compare against.
• Refusal was not re-run on this bf16 build. The parent's numbers carry over (same weights, MLX layout): AdvBench 520 → 1/520 (0.19 %) non-thinking, 0/520 thinking; HarmBench safety categories → 0/293. The oQ4e/oQ6e/oQ8e siblings quantized from it each measured 0/80 on AdvBench and 0/119 on HarmBench-safety (no prefill, non-thinking).

Usage (oMLX)

brew tap jundot/omlx https://github.com/jundot/omlx && brew install jundot/omlx/omlx
hf download windowsxp811203/Qwen3.8-27B-Abliterated-MLX-MTP --include "bf16/*" --local-dir ~/.omlx/models
omlx serve --model-dir ~/.omlx/models     # the model id is the folder name: bf16

Then in ~/.omlx/model_settings.json (note the models wrapper — a top-level model key is silently ignored):

{"version": 1, "models": {"bf16": {"mtp_enabled": true, "mtp_num_draft_tokens": 3}}}

The server log should print Speculative backend selected … Lightning MTP (model_type=qwen3_5, active) when it loads.

Support / 打賞

If these models are useful to you, tips are appreciated — they pay for the GPU time.
如果這些模型對你有幫助，歡迎打賞，用於支應算力成本。

USDT (TRC20) · TPTo32r7vKazpTNaFqfFZ2ztoK1DG88888

Disclaimer

Inherits the Apache-2.0 license of the base model. This model will not refuse; you are responsible for your use.