library_name: mlx
license: apache-2.0
base_model: Qwen/Qwen3.8-27B
pipeline_tag: image-text-to-text
tags:

• mlx
• mlx-vlm
• qwen
• qwen3.8
• multimodal
• 2-bit
• 4-bit
• 6-bit
• 8-bit
• bf16
• abliterated

Qwen3.8-27B Abliterated MLX

An unofficial family of MLX derivatives of
Qwen/Qwen3.8-27B. The original
model is by Qwen; the MLX conversions, refusal-direction experiment, and
validation were performed by PocketAI Model Lab. PocketAiHub identifies the
publisher of these derivatives, not the creator of Qwen3.8.

This consolidated repository contains experimental 2-bit AWQ plus 4-bit,
6-bit, 8-bit, and BF16 variants. It is the canonical download location.

Important safety notice

These checkpoints have been modified to suppress learned refusal behavior. They
may produce harmful, illegal, offensive, deceptive, or dangerously incorrect
content more readily than the upstream instruction model. Abliteration is not
truthfulness training, capability improvement, or a safety guarantee. Use these
models only where you can independently evaluate and constrain their outputs.

Variants

All variants share the pinned upstream revision 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. Quantized
variants use MLX affine quantization; 2-bit uses AWQ/group 32 and 4/6/8-bit use
group 64. Their vision towers remain in BF16.

Abliteration recipe

A projected harmful-minus-harmless direction was measured from 256
length-matched prompts per class at the assistant-generation boundary.

• Direction source layer: 53
• Destination layers: 24–63
• Scale: 1.0
• Per-input-column norm preservation: enabled
• Modified residual-output matrices: 80

See abliteration-manifest.json for the
machine-readable recipe.

Behavioral evaluation

Each variant was evaluated deterministically on 100 harmful prompts and 100
benign controls with batch size 1, thinking disabled, and a 128-token ceiling.

The scorer is a transparent phrase-based explicit-refusal detector. Most
generations reached the token ceiling, so this is an early-refusal screen rather
than a full-completion evaluation. It cannot establish universal compliance or
answer quality. Machine-readable results are in
benchmarks/validation-summary.json
and each variant's validation-summary.json.

Feature validation

The 4-bit, 6-bit, 8-bit, and BF16 variants passed:

• 12/12 deterministic quality checks
• 8/8 native tool-call checks
• text and image smoke tests
• temporal video understanding (red->blue)
• 4K-context retrieval (COBALT-7319)

The 2-bit AWQ build is experimental. It measured 9/12 quality checks,
0/8 tool-call checks, passed vision but not exact-text smoke, missed the
synthetic temporal-video check, and found the 4K needle but did not return it
exactly. These limitations are preserved in the machine-readable evidence.

4K MLX performance

These are single local runs on an Apple M5 Max with 128 GB unified memory,
using mlx==0.32.0, mlx-vlm==0.6.8, batch size 1, temperature 0, seed 0,
and thinking disabled. Each prompt contained 4,105 tokens.

These measurements are not cross-machine performance guarantees.

Download and load

python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"

Download only the desired variant:

from pathlib import Path

from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template

repo_id = "PocketAiHub/Qwen3.8-27B-Abliterated-MLX"
variant = "4bit"  # "2bit", "4bit", "6bit", "8bit", or "bf16"

snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))

prompt = apply_chat_template(
    processor,
    model.config,
    "Explain why seasons occur.",
    num_images=0,
    enable_thinking=False,
)
result = generate(
    model,
    processor,
    prompt,
    max_tokens=256,
    temperature=0.0,
    enable_thinking=False,
)
print(result.text)

Image and video inputs use the normal mlx_vlm.generate media arguments.

License and attribution

The base model is Apache 2.0 licensed. See LICENSE and the
official Qwen model card.