license: apache-2.0
base_model: orcarouter/Qwen3.8-27B-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:

• mlx
• qwen3_5
• image-text-to-text
• mxfp8
• apple-silicon
• uncensored

orcarouter-Qwen3.8-27B-Uncensored — MLX MXFP8

MLX MXFP8 quantization of orcarouter/Qwen3.8-27B-Uncensored,
an uncensored fine-tune of Qwen3.8-27B. Qwen3.8-27B is a qwen3_5 vision-language model with
a hybrid GatedDeltaNet linear-attention + full-attention text backbone (64
layers, full attention every 4th) and a Qwen3-VL vision tower. Runs on Apple
Silicon via mlx-vlm. Stays
image-text-to-text — the vision tower is kept in bf16; only the text backbone
is quantized.

Quantizations

Verification

This higher-fidelity build was verified structurally (correct tensor shapes,
format: mlx metadata, consistent shard index, vision tower intact in bf16). Full
token-by-token generation was not benchmarked on the 24 GB test machine because
27 GB exceeds its RAM; on a 32 GB+ Mac it runs at normal speed. Since MXFP8 uses
more bits than the MXFP4 build — which passed text + vision smoke tests
end-to-end — it is at least as faithful to the base model.

See the MXFP4 build for the
full generation/vision smoke-test results.

Usage (mlx-vlm)

pip install -U mlx-vlm   # needs the qwen3_5 architecture (>= 0.6.12)

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("sahilchachra/orcarouter-Qwen3.8-27B-Uncensored-MXFP8")
config = model.config

prompt = apply_chat_template(processor, config, "What is the capital of France?")
print(generate(model, processor, prompt, max_tokens=256, verbose=True))

This is a reasoning model; it produces a <think> channel before its answer.

Run in LM Studio

Loads and runs in LM Studio (tested on 0.4.20, mlx-llm runtime): the
qwen3_5 architecture is recognized, the model indexes cleanly (format: mlx
present), and the ChatML template runs as-is. This is a reasoning model — it
emits a thinking channel (reasoning_content) before the final content, so give
it enough max_tokens (e.g. 200+) or the answer can be empty while it is still
thinking. Verified: "capital of France" → reasoning + content = "Paris".

Note: at 27 GB this needs a 32 GB+ Mac to load under LM Studio's default guardrails.

Notes & limitations

• Uncensored model. This is a deliberately uncensored/abliterated derivative and
  will not refuse requests the way the original might. Use responsibly and in line
  with the base model's license and your local laws.
• MTP head dropped. The base model's multi-token-prediction (speculative
  decoding) head is not included — mlx-vlm strips the mtp.* weights on load. Text
  and vision are unaffected; only spec-decode is not available.
• tie_word_embeddings=false, so the ~1.27B lm_head is a separate matrix and is
  quantized. Verified correct on the MXFP4 build above.
• Inherits all capabilities and limitations of the base model. See the
  original model card.
• Quantized by @sahilchachra with MLX.
  Apache-2.0.