license: apache-2.0
base_model: abliterant/Qwen3.8-27B-RANA-abliterated
base_model_relation: quantized
library_name: mlx
pipeline_tag: image-text-to-text
tags:

• abliterant
• mlx
• mlx-vlm
• apple-silicon
• abliteration
• refusal-direction
• qwen3
• vision-language
• 4-bit
• 5-bit
• 6-bit
• 8-bit

Qwen3.8-27B-RANA-abliterated-MLX

MLX builds of abliterant/Qwen3.8-27B-RANA-abliterated, a refusal-ablated Qwen/Qwen3.8-27B, for research with Apple Silicon and mlx-vlm.

Quick start · Evaluation · Available files · Abliterant models

This is a safety-alignment-removed research model. Read Intended use and Limitations before using it. Method, full evaluation and release gates are on the BF16 card.

At a glance

Quick start

Download one precision folder, then load it by local path:

pip install -U mlx-vlm    # mlx-vlm >= 0.7.3, mlx >= 0.32.2
hf download abliterant/Qwen3.8-27B-RANA-abliterated-MLX --include "4-bit/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/4-bit \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

Another precision, plus the MTP drafter (repeat --include for each folder):

hf download abliterant/Qwen3.8-27B-RANA-abliterated-MLX --include "8-bit/*" --include "mtp/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit \
  --draft-model ./Qwen3.8-27B-RANA-abliterated-MLX/mtp --draft-kind mtp \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

• Don't pass the repository ID itself to --model. mlx-vlm then downloads every folder (about 160 GB), not just the 4-bit files at the root. The root repeats the 4-bit/ files for apps that open a repository's top level.
• Add --image path/to/image.png for vision input. The drafter is optional; it only speeds up generation.
• An OpenAI-compatible server: python -m mlx_vlm server --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit (add --draft-model …/mtp --draft-kind mtp for speculative decoding). It supports tool calls and returns the reasoning separately.
• Thinking is on by default. Qwen's thinking-mode sampling: temperature 1.0, top_p 0.95, top_k 20.
• Long technical requests can need 20–50k tokens of reasoning; allow a large --max-tokens.

Release checks used mlx-vlm 0.7.3 and MLX 0.32.2 on Linux with MLX's CUDA backend. Apple Silicon performance was not measured.

What changed

• Source: the published BF16 weights, converted with mlx_vlm.convert (mlx-vlm 0.7.3, MLX 0.32.2). Quantized builds use affine round-to-nearest quantization with group size 64; the vision tower stays in BF16. bf16/ is the same conversion without quantization. The MTP head is split into mtp/ with --mtp and kept in BF16. This is a conversion and quantization of the refusal-ablated weights, not additional training.
• Layout: in every folder, tensor names, shapes and dtypes and config.json are identical to the matching mlx-community or lmstudio-community build of Qwen/Qwen3.8-27B (4-bit/ and 8-bit/: mlx-community -4bit / -8bit; 5-bit/ and 6-bit/: lmstudio-community -MLX-5bit / -MLX-6bit; bf16/: mlx-community -bf16; mtp/: mlx-community -MTP-bf16). It loads wherever those load.

The RANA abliteration method, evaluation and release gates are documented on the BF16 card. The MLX conversion record does not specify an exact source BF16 revision.

Evaluation

All recorded checks ran with MLX 0.32.2's CUDA backend on Linux, using files sha256-identical to the evaluated release artifacts. Raw outputs are in results/.

KLD is the mean KL divergence of each build's next-token distribution from the bf16/ build on wiki.test.raw, using the same protocol as the GGUF card: 100 chunks of 512 tokens, with the second half of each chunk scored. Lower KLD is closer to BF16. Each build has 25,500 scored tokens. This measures quantization drift, not refusal behavior.

• Files: every release file was hashed after upload (sha256 for weights) and matched the conversion output.
• Function: with mlx_vlm server, reading text and a shape from an image and a three-turn tool-calling exchange passed on 4-bit/, 8-bit/ and bf16/.
• MTP speculative decoding: with mtp/, greedy decoding and 512 tokens of a coding answer, mean accepted tokens per round were: 4-bit 2.49 (74% of drafted), 5-bit 2.57 (79% of drafted), 6-bit 2.56 (78% of drafted), 8-bit 2.57 (79% of drafted), bf16 2.57 (79% of drafted).
• Quick-start checks: the release commands were run with --max-tokens 4096, including the repeated---include download of 8-bit/ and mtp/.

Available files

The root is the 4-bit build; other precisions are in subfolders, and mtp/ holds the drafter for MTP speculative decoding. Vision is included in every model build.

Other formats: BF16 (Transformers), FP8 (vLLM / SGLang), GGUF (llama.cpp).

Limitations and intended use

Limitations

• Not tested on Apple Silicon in the recorded release checks. The checks above ran MLX's CUDA backend on Linux (same MLX version, same model code, same files); speed on a Mac is not measured here.
• Refusal behavior was not re-measured on the MLX builds. KLD shows how closely each build tracks the BF16 weights; refusal results for BF16, FP8 and a 4-bit GGUF are on their cards. Lower-bit builds drift further from BF16.
• Everything listed under Limitations on the BF16 card applies here too.
• Reduced refusal does not establish greater safety, accuracy, capability retention or universally compliant behavior.

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behavior.
• Not for public or end-user deployment without a separate moderation layer. The model's own refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen, and the terms of any platform where outputs are used.

Provenance and license

These MLX artifacts derive from the Abliterant BF16 release of Qwen/Qwen3.8-27B. The release evidence is recorded at revision 8a647c2e5e329dfdd399c9541b8058fa5447c228.

The weights retain the Apache-2.0 license inherited from Qwen. Conversion, quantization and refusal ablation do not remove the upstream license obligations.

Credits:

• Qwen team: base model Qwen/Qwen3.8-27B.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• Apple MLX team and mlx-vlm (Prince Canuma): MLX and the conversion and runtime tooling.