language: en
library_name: mlx
license: apache-2.0
model_name: Qwen3.8-9B Distill Abliterated MLX
base_model: empero-ai/Qwen3.8-9B
pipeline_tag: image-text-to-text
tags:
- mlx
- mlx-vlm
- qwen
- qwen3.8
- distillation
- multimodal
- abliterated
- 4-bit
- 8-bit
- bf16
Qwen3.8-9B Distill Abliterated MLX
Abliterated MLX derivatives of the third-party distillempero-ai/Qwen3.8-9B, pinned to
revision 0934f3d2327ff2df2197495278c4c46ae5a56bd9. The source is a third-party full-parameter
distillation based on Qwen/Qwen3.5-9B; it
is not an official Qwen3.8 release. Abliteration, conversion, and validation
were performed by PocketAI Model Lab.
Important safety notice
These checkpoints were intentionally modified to suppress learned refusal
behavior. They may respond more readily to requests involving potentially
unsafe, illegal, offensive, deceptive, or dangerously incorrect content.
Abliteration is not truthfulness training or a safety guarantee. Independently
constrain and evaluate outputs for the intended deployment.
Variants
| Precision | Folder | Packaged size |
|---|---|---|
| 4-bit | 4bit/ |
5.57 GiB |
| 8-bit | 8bit/ |
9.74 GiB |
| BF16 | bf16/ |
17.55 GiB |
The 4-bit and 8-bit variants use MLX affine quantization with group size 64.
The vision tower remains BF16. The BF16 variant is unquantized. Native source
MTP tensors are intentionally excluded.
Refusal-behavior screen
All variants were evaluated on 100 refusal-elicitation test prompts and 100
benign controls in non-thinking mode with deterministic decoding and a
256-token ceiling.
| Precision | Test-set explicit refusals | Control explicit refusals | Test-set natural stops | Control natural stops |
|---|---|---|---|---|
| 4-bit | 0/100 | 0/100 | 11/100 | 2/100 |
| 8-bit | 0/100 | 0/100 | 14/100 | 4/100 |
| BF16 | 0/100 | 0/100 | 11/100 | 3/100 |
The transparent phrase-based screen found no explicit refusals or evasive
non-answers, and every case contained final-answer text. Most generations hit
the 256-token ceiling, so this is an early-refusal regression screen—not proof
of universal compliance, safety, factuality, or completion quality.
KV/long-context evaluation
| Precision | Formatted tokens | Prefill tok/s | Decode tok/s | Peak MLX memory |
|---|---|---|---|---|
| 4-bit | 65,536 | 1028.1 | 44.59 | 13.05 GB |
| 8-bit | 32,776 | 2323.7 | 52.99 | 14.19 GB |
| BF16 | 32,776 | 2299.3 | 27.32 | 22.67 GB |
All three exact-retrieval cases passed. The 8-bit and BF16 runs used 16-bit KV
cache quantization at 32K; the 4-bit run was an unquantized-KV 64K text test.
The 4-bit and BF16 variants also passed the complete deterministic 4K feature
suite; all three passed text and vision runtime smoke tests.
Exact evidence hashes and test details are in each variant'svalidation-summary.json and artifact-manifest.json.
Download and load
python -m pip install "mlx==0.32.0" "mlx-vlm==0.6.8"
from pathlib import Path
from huggingface_hub import snapshot_download
from mlx_vlm import generate, load
from mlx_vlm.prompt_utils import apply_chat_template
repo_id = "PocketAiHub/Qwen3.8-9B-Abliterated-MLX"
variant = "4bit" # "4bit", "8bit", or "bf16"
snapshot = Path(snapshot_download(repo_id, allow_patterns=[f"{variant}/*"]))
model, processor = load(str(snapshot / variant))
prompt = apply_chat_template(
processor,
model.config,
"Explain why seasons occur.",
num_images=0,
enable_thinking=False,
)
result = generate(
model,
processor,
prompt,
max_tokens=256,
temperature=0.0,
enable_thinking=False,
)
print(result.text)
Reproducibility and limitations
- Source:
empero-ai/Qwen3.8-9Bat0934f3d2327ff2df2197495278c4c46ae5a56bd9 - Declared base:
Qwen/Qwen3.5-9B - Abliteration is a targeted directional intervention, not general evaluation
- Standard MLX conversion intentionally excludes native MTP tensors
- This is an experimental community release; verify behavior for your use case
License and attribution
The source repository declares Apache-2.0. This derivative includes the Apache
2.0 text in LICENSE. Original model credit remains with Empero
and the Qwen team; PocketAI is the derivative publisher.