license: apache-2.0
base_model: preemware/Qwen3.8-27B-RANA-abliterated
base_model_relation: quantized
library_name: mlx
pipeline_tag: image-text-to-text
tags:

• mlx
• mlx-vlm
• apple-silicon
• abliteration
• refusal-direction
• qwen3
• vision-language
• 4-bit
• 5-bit
• 6-bit
• 8-bit

Qwen3.8-27B-RANA-abliterated-MLX

MLX builds of preemware/Qwen3.8-27B-RANA-abliterated,
a refusal-ablated Qwen/Qwen3.8-27B, for Apple Silicon (MLX, mlx-vlm).
One repo: the root is the 4-bit build, the other precisions are in subfolders, and mtp/ holds the
drafter for MTP speculative decoding. Vision is included in every build.

This is a safety-alignment-removed research model. Read Intended use and
Limitations before using it. Method, full evaluation and release gates are on the
BF16 card.

Builds

KLD = mean KL divergence of each build's next-token distribution from the bf16/ build on wiki.test.raw, the
same protocol as the GGUF card: 100 chunks of
512 tokens, second half of each chunk scored. Lower is closer to BF16.

Other formats: BF16 (Transformers),
FP8 (vLLM / SGLang),
GGUF (llama.cpp).

Quick start

Download one precision folder, then load it by path:

pip install -U mlx-vlm    # mlx-vlm >= 0.7.3, mlx >= 0.32.2
hf download preemware/Qwen3.8-27B-RANA-abliterated-MLX --include "4-bit/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/4-bit \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

Another precision, plus the MTP drafter (repeat --include for each folder):

hf download preemware/Qwen3.8-27B-RANA-abliterated-MLX --include "8-bit/*" --include "mtp/*" \
  --local-dir ./Qwen3.8-27B-RANA-abliterated-MLX

python -m mlx_vlm generate --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit \
  --draft-model ./Qwen3.8-27B-RANA-abliterated-MLX/mtp --draft-kind mtp \
  --enable-thinking --max-tokens 32768 --temperature 1.0 --top-p 0.95 --top-k 20 \
  --prompt "Explain how a hash map handles collisions."

• Don't pass the repo id itself to --model. mlx-vlm then downloads every folder (about 160 GB), not just
  the 4-bit files at the root. The root copy is there for apps that open a repo's top level.
• Add --image path/to/image.png for vision input. The drafter is optional; it only speeds up generation.
• An OpenAI-compatible server: python -m mlx_vlm server --model ./Qwen3.8-27B-RANA-abliterated-MLX/8-bit (add
  --draft-model …/mtp --draft-kind mtp for speculative decoding). It supports tool calls and returns the reasoning separately.
• Thinking is on by default. Qwen's thinking-mode sampling: temperature 1.0, top_p 0.95, top_k 20.
• Long technical requests can need 20–50k tokens of reasoning; allow a large --max-tokens.

Intended use

• Research only: interpretability, red-teaming, and robustness evaluation of refusal behavior.
• Not for public or end-user deployment without a separate moderation layer. The model's own
  refusals have been largely removed, so any safety filtering has to happen outside it.
• You are responsible for complying with applicable law, the Apache-2.0 license inherited from Qwen,
  and the terms of any platform where outputs are used.

How it was made

• Source: the published BF16 weights, converted with mlx_vlm.convert (mlx-vlm 0.7.3, MLX 0.32.2).
  Quantized builds use affine round-to-nearest quantization with group size 64; the vision tower stays in BF16.
  bf16/ is the same conversion without quantization. The MTP head is split into mtp/ with --mtp and kept in BF16.
• Same layout as the standard conversions of the base model: in every folder, tensor names, shapes and dtypes and
  config.json are identical to the matching mlx-community or lmstudio-community build of Qwen/Qwen3.8-27B
  (4-bit/ and 8-bit/: mlx-community -4bit / -8bit; 5-bit/ and 6-bit/: lmstudio-community -MLX-5bit / -MLX-6bit; bf16/: mlx-community -bf16; mtp/: mlx-community -MTP-bf16). It loads wherever those load.

Checks

All checks ran with MLX 0.32.2's CUDA backend on Linux, on files sha256-identical to the ones in this repo. Raw outputs are in results/.

• Files: after upload, every file here was hashed (sha256 for weights) and matched the conversion output.
• Quality: the KLD table above, 25,500 scored tokens per build. Perplexity on the same tokens: BF16 6.778, 4-bit 6.945, 5-bit 6.863, 6-bit 6.788, 8-bit 6.779.
• Function: with mlx_vlm server, reading text and a shape from an image and a three-turn tool-calling exchange pass on 4-bit/, 8-bit/ and bf16/.
• MTP speculative decoding with mtp/, greedy, 512 tokens of a coding answer; mean accepted tokens per round: 4-bit 2.49 (74% of drafted), 5-bit 2.57 (79% of drafted), 6-bit 2.56 (78% of drafted), 8-bit 2.57 (79% of drafted), bf16 2.57 (79% of drafted).
• The Quick start commands were run as written against this repo (with --max-tokens 4096), including the repeated---include download of 8-bit/ and mtp/.

Limitations

• Not run on Apple Silicon by us. The checks above ran MLX's CUDA backend on Linux (same MLX version,
  same model code, same files); speed on a Mac is not measured here.
• Refusal behavior was not re-measured on the MLX builds. KLD shows how closely each build tracks the
  BF16 weights; refusal results for BF16, FP8 and a 4-bit GGUF are on their cards. Lower-bit builds
  drift further from BF16.
• Everything listed under Limitations on the BF16 card
  applies here too.

Credits

• Qwen team: base model Qwen/Qwen3.8-27B.
• Arditi et al., 2024: "Refusal in Language Models Is Mediated by a Single Direction".
• Jim Lai (grimjim): prior work on norm-preserving abliteration.
• Apple MLX team and mlx-vlm (Prince Canuma): MLX and the conversion and runtime tooling.