license: apache-2.0
base_model: Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
tags:

• qwen3.8
• qwen
• 27b
• dense
• vision-language
• multimodal
• abliterated
• quantized
• tool-calling
• long-context
• mlx
  pipeline_tag: image-text-to-text
  library_name: mlx

Qwen3.8-27B-ABLITERATED-mlx-q3km

MLX conversion of Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16,
quantized with the mixed_3_4 recipe as an equivalent to GGUF Q3_K_M.

• 3.910 bits per weight, 13.4 GB across 3 safetensors shards
• Architecture qwen3_5 (Qwen3_5ForConditionalGeneration), multimodal, hybrid
  linear/full attention every 4th layer
• Requires mlx-vlm >= 0.6.13 (earlier versions have no qwen3_5 module)

Why mixed_3_4 and not --q-bits 3

MLX's mixed-bit predicates are a deliberate port of llama.cpp's _K_M scheme,
not an approximation. Both give more bits to v_proj and down_proj in the
first eighth of layers, the last eighth, and every third layer between, plus a
high-bit lm_head. mixed_3_4 is 3-bit base with 4-bit on those sensitive
tensors, which is exactly Q3_K_M.

The measured result confirms it: 3.910 bpw here, against 3.91 bpw for the
official GGUF Q3_K_M (13.30 GB for ~27.2B params). A plain --q-bits 3 would
land near 3.5 bpw and would not match.

Note that mlx-vlm never quantizes the vision encoder at any level, so the
vision tower stays bf16.

Conversion

mlx_vlm.convert \
  --hf-path Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16 \
  --mlx-path ./Qwen3.8-27B-ABLITERATED-mlx-q3km \
  --quantize --quant-predicate mixed_3_4

Usage

pip install -U mlx-vlm

# text
mlx_vlm.generate --model wfiedler/Qwen3.8-27B-ABLITERATED-mlx-q3km \
  --prompt "Explain unified memory in two sentences." --max-tokens 256

# vision
mlx_vlm.generate --model wfiedler/Qwen3.8-27B-ABLITERATED-mlx-q3km \
  --prompt "Read this receipt and list every line item with its price." \
  --image receipt.png --max-tokens 400

Quality vs the 5-bit conversion

Benchmarked against the q5 build (5.678 bpw, 18 GB) of the same base model, on
images with exact ground truth, 12 tasks x 3 trials at temperature 0.7.

On reading images the two are indistinguishable. Every OCR-shaped task ties
perfectly: all receipt fields, all chart values, all table cells, every rare
token in the small print.

The one real gap is counting many objects. Given 20 scattered shapes in four
colour/type groups, q5 answered correctly three times out of three; this model
answered 17, 19 and 21, never correct. The difference is visible in the output:
q5 enumerates shape by shape before totalling, while the 3-bit build emits a
terse number. The low-bit quant lost the habit of decomposing before answering.

Neither model does reliable mental arithmetic without a scratchpad; both compute
a four-item subtotal wrongly at temperature 0.7.

Measured on an Apple Silicon Mac with 64 GB unified memory, each model in its
own process. (GenerationResult.peak_memory is a process-global high-water mark
that does not reset between models, so comparing two in one process makes the
second inherit the first's peak.)

Use this build for OCR, document and chart work — 46% faster and 6.6 GB
lighter for no measurable loss. Use the 5-bit build when the task involves
counting or enumerating things in a scene.

Provenance and caveats

This is an abliterated model: the base has had refusal behaviour removed by its
authors. It will comply with requests a stock instruct model would decline.
Evaluate it before using it anywhere user-facing.

Quantization is lossy. The benchmark above covers a narrow slice of behaviour
and says nothing about long-context, tool-calling, or code quality at 3 bits.
All credit for the model itself goes to Blackfrost-AI and the Qwen team.