base_model: huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated
base_model_relation: quantized
model_name: Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp
library_name: mlx
pipeline_tag: image-text-to-text
license: apache-2.0
tags:

• mlx
• omlx
• quantization
• 8-bit
• apple-silicon
• mtp
• speculative-decoding
• qwen
• qwen3.5
• vision
• fp16
• abliterated
• uncensored
• derivative

Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp

Unofficial community quantization. This repository is not affiliated
with or endorsed by huihui.ai, empero-ai, Alibaba Cloud/Qwen, or oMLX.

An MLX/oMLX 8-bit quantization of
huihui-ai/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated
for Apple Silicon. Language-model weights are 8-bit affine (group size 64).
Retained non-quantized tensors are FP16. The vision tower and the MTP draft
head are kept. This is a quantization/conversion, not a finetune. No
broad accuracy evaluation was performed on this checkpoint.

Content warning: the source model is abliterated: its refusal behaviour
was deliberately removed. It can produce harmful, offensive, or inaccurate
content. The upstream usage warnings in SOURCE_MODEL_CARD.md apply in
full. Use it for research and in controlled settings only.

Source and conversion

The immediate source is huihui-ai's BF16 abliterated checkpoint at revision
efcc73cac15ff8fc5d46b8d41b53c22d571cf97d. That checkpoint is an abliteration of
empero-ai/Qwythos-9B-Claude-Mythos-5-1M
(revision 14a29bae5143091aeaf87ad37120de4cd57d592c), which is in turn a
finetune of Qwen/Qwen3.5-9B.
This checkpoint was not requantized from the GGUF release.

Conversion on 2026-10-13 used oMLX 0.7.0's streaming oQ quantizer, with
level 8, group size 64, dtype=float16, preserve_mtp=true, vision
retained, and enhanced/oQe mode disabled. FP16 instead of BF16 targets oMLX's
faster FP16 prefill path on M1/M2 Macs. Every floating-point tensor was
checked after conversion; none contained inf or NaN values.

The three safetensors shards index 1,289 tensors: 699 FP16, 257 packed U32,
and 333 FP32. 29 of them are MTP tensors. The source download took about
2.5 minutes and the conversion about 1.5 minutes on a Mac Studio M2 Ultra.

Context length: 262k default, optional YaRN 1M config

The source repository's config.json enables YaRN (factor 4) for a 1M-token
window. With that config as published, mlx-vlm 0.x failed to load the model:
rope_parameters lacked partial_rotary_factor. The quantization therefore
used the source's own config.json.pre_yarn; the weights are identical.

• config.json: native 262,144-token RoPE config (default).
• config.json.yarn-1m: YaRN factor 4 / 1,048,576 tokens, with the missing
  partial_rotary_factor added. To use it, swap it in as config.json.
  mlx-lm's text loader applies the YaRN scaling. mlx-vlm's qwen3_5 loader
  (oMLX's VLM path) loads it but ignores the scaling, so that path is
  effectively limited to the native window. 1M-token prefill was not tested.

Use with oMLX

Download the complete repository into the oMLX model directory, then restart
or reload oMLX:

hf download jongstra/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp \
  --local-dir ~/.omlx/models/Huihui-Qwythos-9B-Claude-Mythos-5-1M-abliterated-oQ8-fp16-mtp

This repository contains MLX safetensors, not GGUF.

Local smoke test

On a Mac Studio M2 Ultra (64 GB), both mlx_lm and mlx_vlm (as bundled
with oMLX 0.7.0) loaded both configs and gave coherent answers to a short
prompt. A plain mlx_lm run without MTP decoded 400 tokens at about
67 tok/s, with 9.7 GB peak memory. These figures come from one local run;
they are not a benchmark.

License and attribution

The source chain is published under the Apache License 2.0 (Qwen3.5-9B →
empero-ai Qwythos → huihui-ai abliterated). LICENSE contains the Apache
2.0 text. CHANGES.md describes this conversion's modifications, as Section
4(b) of that license requires. The upstream model card is kept verbatim in
SOURCE_MODEL_CARD.md.