license: apache-2.0
base_model: AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
language:
- en
- zh
- multilingual
tags: - mlx
- mlx-vlm
- qwen3_5
- qwen3.8
- apple-silicon
- quantized
- 6-bit
- multimodal
- vision
- vision-language
- gated-deltanet
- hybrid-attention
- mtp
- speculative-decoding
- abliterated
- uncensored
- conversational
Qwen3.8-27B-AEON-Ultimate-Uncensored — Multimodal MLX 6-bit
A 6-bit MLX quantization of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 (revision 8f76e82), AEON's abliterated release of Qwen/Qwen3.8-27B, for local inference on Apple Silicon.
The vision tower is fully preserved (333/333 tensors — AEON's release keeps it hash-identical to stock Qwen3.8), so image and video understanding work through mlx-vlm. The model's native MTP head is published separately as a drafter for lossless speculative decoding: intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter — pairing it typically speeds decode 1.4–1.9× at identical output quality.
Requirements
- Apple Silicon Mac with MLX support;
pip install mlx-vlm(converted with mlx-vlm 0.6.13 / mlx 0.32.0). - Unified memory: ~21 GB for weights, with an observed runtime peak of ~25 GiB at short context and ~30 GiB at 13k-token context. Comfortable on 48 GB+ machines, workable on 36 GB; not recommended below that.
- Because decode is memory-bandwidth-bound, tokens/sec scales roughly linearly with the chip's memory bandwidth (see measured numbers below).
Usage
# text / vision
python -m mlx_vlm generate \
--model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
--prompt "Describe this image." --image photo.jpg
# faster decode with the MTP drafter (lossless speculative decoding)
python -m mlx_vlm generate \
--model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-Multimodal-MLX-6bit \
--draft-model intheblue/Qwen3.8-27B-AEON-Ultimate-Uncensored-MLX-MTP-Drafter \
--draft-kind mtp --draft-block-size 3 \
--prompt "Write a short story about a lighthouse keeper."
Tips:
- Draft block size 3 is the all-round sweet spot; 4 edges ahead on code; ≥5 regresses.
- When serving (
mlx_vlm.server),--prefill-step-size 512cuts peak prefill memory by ~6 GB at no measured speed cost. - Recommended sampling (from the Qwen3.8 card): thinking
temp=1.0, top_p=0.95, top_k=20; non-thinkingtemp=0.7, top_p=0.8, presence_penalty=1.5.
Conversion recipe
python -m mlx_vlm convert \
--hf-path AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
-q --q-bits 6 --q-group-size 64 \
--mlx-path Qwen3.8-27B-AEON-Ultimate-Uncensored-mlx-6bit
Affine mode, group size 64, no calibration (RTN is near-lossless at 6-bit). Language model and vision tower are both quantized at the global setting; mtp.* tensors are excluded by design — mlx-vlm loads the MTP drafter as a separate model (see the drafter repo for the split recipe).
Measured performance
Test machine: Mac mini M4 Pro, 48 GB unified memory (~273 GB/s). Higher-bandwidth chips (M-series Max/Ultra) scale decode proportionally — serial decode on this quantization measures at ~100% of the device's streaming-bandwidth roofline.
| Workload | Config | Decode |
|---|---|---|
| Coding (temp 0.2) | MTP block 4 | 21.5 tok/s |
| Document QA @ 13k ctx (temp 0.7) | MTP block 3 | 16.9 tok/s |
| Creative prose (temp 0.7) | MTP block 3 | 15.9 tok/s |
| Serial (no drafter) | — | 11.4 tok/s |
Prefill measured ~105–110 tok/s, flat with context length up to the 13k tested. Draft acceptance is workload-dependent: ~46% on open-ended prose, substantially higher on code and grounded QA.
Validation
- Text, vision, and long-context (13k) generation smoke-tested after conversion.
- Two-phase coding gate (spec research → implementation, executed against 28 hidden edge-case asserts across an SSE-parser task and a stack-VM task): 28/28.
- Speculative decoding verified lossless (drafter rejections fall back to the target model's own tokens; the sampling distribution is unchanged by construction).
Provenance & responsibility
Qwen/Qwen3.8-27B → AEON-7 SSM-conv1d repair + abliterix abliteration (BF16, vision and MTP untouched — see their card for methodology and KL evidence) → this repo (6-bit MLX quantization, nothing else changed).
This is an abliterated, refusal-removed model. As the upstream card puts it: the model does not decide whether to comply — you do. Outputs are the responsibility of the operator; use within the law of your jurisdiction. Apache-2.0, inherited from base.