license: apache-2.0
library_name: mlx
pipeline_tag: image-text-to-text
base_model:

• huihui-ai/Huihui-Qwen3.8-27B-abliterated
  base_model_relation: quantized
  tags:
• qwen3
• qwen3.8
• mlx
• mlx-vlm
• 6-bit
• abliterated
• uncensored
• vision

daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-6bit

This is an unofficial MLX 6-bit conversion of
huihui-ai/Huihui-Qwen3.8-27B-abliterated,
prepared for local inference on Apple silicon with
mlx-vlm.

The conversion is pinned to upstream revision
739e3c5b89849f6c238ce1e5b70008612ae42cdd.
No additional fine-tuning or abliteration was performed during conversion.

What is included

• Qwen3_5ForConditionalGeneration architecture in MLX Safetensors format.
• Language-model weights quantized to 6-bit affine RTN with group size 64.
• Vision components and multimodal processors retained at their source precision.
• Five weight shards, approximately 23 GB (21 GiB) in total.
• Text and image-to-text inference support through mlx-vlm.
• No standalone MTP drafter and no usable mtp.* checkpoint tensors.

The upstream model reports that layers 18 through 51 were ablated while its
visual components were left unmodified. See the
upstream model card
for details about the abliteration procedure and the original model.

Conversion details

Equivalent conversion settings:

python -m pip install "mlx-vlm==0.6.16" "mlx==0.32.2" jinja2

mlx_vlm.convert \
  --hf-path huihui-ai/Huihui-Qwen3.8-27B-abliterated \
  --revision 739e3c5b89849f6c238ce1e5b70008612ae42cdd \
  --mlx-path ./Huihui-Qwen3.8-27B-abliterated-MLX-6bit \
  --quantize \
  --q-bits 6 \
  --q-group-size 64 \
  --q-mode affine \
  --quant-method rtn

Usage with mlx-vlm

Install a recent mlx-vlm. Version 0.6.16 is the version used and tested for
this conversion.

python -m pip install -U "mlx-vlm>=0.6.16"

The examples below use the full Hugging Face repository ID. A local model
directory can be used instead.

Text generation

MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-6bit"

mlx_vlm.generate \
  --model "$MODEL" \
  --prompt "Explain why the sky appears blue." \
  --max-tokens 512 \
  --temperature 1.0 \
  --top-p 0.95 \
  --top-k 20

Image understanding

MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-6bit"

mlx_vlm.generate \
  --model "$MODEL" \
  --image /absolute/path/to/image.jpg \
  --prompt "Describe this image in detail." \
  --max-tokens 512 \
  --temperature 1.0 \
  --top-p 0.95 \
  --top-k 20

Python example

from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

model_path = "daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-6bit"
image_path = "/absolute/path/to/image.jpg"

model, processor = load(model_path)
messages = [
    {
        "role": "user",
        "content": [{"type": "text", "text": "Describe this image in detail."}],
    }
]
prompt = apply_chat_template(
    processor,
    model.config,
    messages,
    num_images=1,
)

result = generate(
    model,
    processor,
    prompt,
    image=[image_path],
    max_tokens=512,
    temperature=1.0,
    top_p=0.95,
    top_k=20,
)
print(result.text)

The bundled generation_config.json uses temperature=1.0, top_p=0.95, and
top_k=20. These are useful starting points rather than mandatory settings.

Recommended DFlash2 draft model

For speculative decoding, the recommended pairing is
z-lab/Qwen3.8-27B-DFlash2,
an Apache-2.0 DFlash 2 draft checkpoint for Qwen/Qwen3.8-27B. It is a separate
dependency and is not included in this repository. The draft is not a
standalone language model; a compatible runtime uses it to propose tokens that
are verified by this target model.

The published draft checkpoint uses a block size of 8 and a sliding window of
2048. The same draft has been locally compatibility-tested with the 4-bit
sibling built from this exact source revision. This 6-bit release has not been
separately inference-tested. Because the target weights differ from the original
Qwen3.8-27B, acceptance rates vary by prompt and workload. A compatible
speculative runtime still verifies output against the target model, but the
speedup is runtime- and workload-dependent.

Use with mlx-vlm

mlx-vlm 0.6.16 can load this DFlash2 checkpoint directly:

MODEL="daguoagi/Huihui-Qwen3.8-27B-abliterated-MLX-6bit"

mlx_vlm.generate \
  --model "$MODEL" \
  --prompt "Explain why the sky appears blue." \
  --max-tokens 512 \
  --temperature 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --draft-model z-lab/Qwen3.8-27B-DFlash2 \
  --draft-kind dflash \
  --draft-block-size 8

In mlx-vlm 0.6.16, the draft checkpoint is loaded at its native precision;
that version does not expose the runtime draft-quantization controls described
below for oMLX.

Reference oMLX configuration

The following production configuration was tested with oMLX 0.6.3rc3 and the
4-bit sibling model. It is provided as a starting point for this 6-bit release:

On the tested 4-bit sibling, text requests use the DFlash2 path while image
requests retain the VLM fallback path. This 6-bit release has not been separately
validated with DFlash2. The draft model and its license should be reviewed and
downloaded separately. Local throughput results are intentionally not presented
as portable performance claims because they depend heavily on the Apple silicon
device, runtime implementation, context length, and draft acceptance rate.

Conversion validation

Per the release workflow, this 6-bit checkpoint was not subjected to text,
vision, DFlash2, or performance inference tests. Release preparation validates
only the conversion artifacts:

• The mlx-vlm 0.6.16 conversion completed successfully.
• The converter reported an average of 6.661 bits per weight.
• Configuration files and the Safetensors index parse successfully.
• All five weight shards referenced by the index are present.
• The Hugging Face model-card metadata parses successfully.

These checks establish artifact completeness, not model quality or runtime
compatibility on every MLX version.

Limitations

• 6-bit quantization can reduce quality relative to the source checkpoint.
• The model inherits the behavior and limitations of the upstream abliterated
  model. The uncensored or abliterated label does not guarantee that every
  refusal has been removed, nor does it guarantee factual or safe output.
• Vision weights were retained, but multimodal quality has not been evaluated
  with a formal benchmark suite.
• MTP weights are not included in this repository.
• DFlash2 acceleration, when used, requires a separate compatible draft model
  and runtime; it is not bundled with this checkpoint.
• This checkpoint targets Apple silicon and MLX. It is not a drop-in replacement
  for the original Transformers checkpoint.

Users are responsible for evaluating outputs and ensuring that their use
complies with applicable laws, policies, and the upstream model license.

License and attribution

The upstream model is released under the Apache License 2.0. This conversion
retains that license. Please also review the
huihui-ai/Huihui-Qwen3.8-27B-abliterated
and Qwen/Qwen3.8-27B model cards.