license: apache-2.0
base_model: huihui-ai/Huihui-Qwen3.8-27B-abliterated
base_model_relation: quantized
pipeline_tag: text-generation
library_name: mlx
tags:

• mlx
• mtplx
• apple-silicon
• speculative-decoding
• mtp
• multi-token-prediction
• abliterated
• uncensored
• qwen3
• not-for-all-audiences

Huihui-Qwen3.8-27B-abliterated-MTPLX-Q8

An MLX INT8 build of huihui-ai/Huihui-Qwen3.8-27B-abliterated
with its native Multi-Token Prediction (MTP) draft head kept intact and packed for
self-speculative decoding on Apple Silicon.

On an M5 Max this decodes at 54.0 tok/s at draft depth 3 versus 17.1 tok/s autoregressive —
a 3.17x speedup with no separate draft model, because the MTP head ships inside the model.

This model is uncensored. Its safety alignment has been removed. See
Intended use and risks before you deploy it.

Provenance

This build is two transformations away from the original Qwen release, and neither of them
trained anything:

1. Qwen/Qwen3.8-27B — the original release.
2. huihui-ai/Huihui-Qwen3.8-27B-abliterated (commit 739e3c5b89) — an abliteration
   of the above, produced with
   Sumandora/remove-refusals-with-transformers.
   Abliteration is a direct weight edit that identifies the model's refusal direction in
   activation space and projects it out. It is not fine-tuning and uses no training data.
   Only layers 18-51 of 64 were ablated; the MTP draft head and the vision tower were
   left unmodified.
3. This repo — an MTPLX forge of the above: INT8 body quantization plus a structurally
   quantized MTP sidecar. No calibration data, no training, no gradient ever computed.

Because huihui's edit never touched the MTP head, this build's draft head is bit-for-bit the
head from the original Qwen release. That is verified, not assumed: see
On the sidecar.

Build recipe

The trunk was forged with MTPLX using mtp_policy: keep_bf16:

{
  "body_bits": 8,
  "body_group_size": 64,
  "body_mode": "affine",
  "mtp_policy": "keep_bf16"
}

The MTP sidecar was then quantized post hoc as a separate structural step, matching how
the official MTPLX reference build was made:

On the sidecar

The quantized sidecar in this repo is byte-identical (SHA-256 1e350715…) to the MTP
sidecar shipped in the official Qwen3.8-27B-MTPLX-Optimized-Quality reference build.

That single fact confirms both halves of the provenance claim above: the huihui abliteration
genuinely left the MTP head untouched, and this repack reproduces the official head
quantization procedure exactly rather than approximating it.

Measured performance

Hardware: Apple M5 Max, 128 GB, macOS 26.5.2. Measured with mtplx forge verify
(mtplx 2.9.1, suite long-code-uncapped, max_tokens 2048, profile sustained, fans
pinned). Single run per depth.

Verdict mtp_depth_wins; nothing was flagged as quality-rejected or acceptance-collapsed.
Depth 3 is the shipped default.

The INT8 sidecar was gated against the BF16 sidecar it replaces, on the same machine and the
same harness. It did not regress — it measured better on every axis:

Both sidecars carry identical draft weights up to INT8 rounding, so treat the gap as
single-run variance on a stochastic decode rather than as evidence that quantizing improved
the head. The honest reading is flat or better, and 398 MB smaller.

What has not been measured

No capability, accuracy, or quality benchmark has been run on this model. The numbers
above are decode throughput and draft-acceptance rates only. Neither the INT8 body
quantization's effect on output quality nor the abliteration's effect on capability has been
evaluated here. If quality matters for your use, benchmark it yourself.

Usage

Requires MTPLX on Apple Silicon; the MTP sidecar is
what makes the speedup work, and a generic MLX loader will ignore it.

mtplx setup --model mvid/Huihui-Qwen3.8-27B-abliterated-MTPLX-Q8 --download
mtplx quickstart --port 8000

Sampler shipped in mtplx_runtime.json: temperature 0.6, top_p 0.95, top_k 20;
draft depth 3, profile sustained.

Roughly 28 GB on disk, so budget a 64 GB machine for comfort at long context.

Intended use and risks

This model is abliterated: the refusal direction has been projected out of its weights.
It will not reliably decline harmful, illegal, unethical, or dangerous requests, and it will
often comply fluently and confidently with requests that the original Qwen3.8-27B would have
refused. The safety behavior is removed from the weights, so it cannot be recovered with a
system prompt.

Consequences worth being explicit about:

• Do not deploy this to untrusted or public users without your own external moderation layer.
• Removing refusals does not add knowledge. Confidently-worded dangerous output can also
  simply be wrong, which makes it more dangerous, not less.
• Abliteration is a blunt instrument. It can degrade instruction-following and reasoning in
  ways unrelated to safety, and as noted above, that has not been measured here.
• You are responsible for what you generate with it and for complying with the Apache-2.0
  terms and applicable law.

It is published for research on alignment, refusal mechanisms, and speculative decoding, and
for adults who understand the above and want an unfiltered local assistant.

License

Apache-2.0, inherited from both parents: Qwen/Qwen3.8-27B and
huihui-ai/Huihui-Qwen3.8-27B-abliterated. Neither the abliteration nor this quantization
adds licensing terms.

Attribution

Powered by MTPLX / https://github.com/youssofal/MTPLX

Thanks to the Qwen team for the base model and the MTP head, to
huihui-ai for the abliteration, and to
Sumandora for the refusal-removal tooling.