base_model: orcarouter/Qwen3.8-Flash-Next-Uncensored
base_model_relation: quantized
library_name: mlx-serve
license: apache-2.0
pipeline_tag: image-text-to-text
tags:

• mlx
• mlx-serve
• apple-silicon
• qwen4-exp
• uncensored
• abliterated
• vision-language

Qwen3.8-Flash-Next Uncensored — MLX-Serve 4-bit: How to Run Locally

The uncensored Qwen3.8-Flash-Next (125B total / 6B active MoE) packed for
mlx-serve, so it actually fits and flies
on a 128 GB Apple Silicon Mac:

• ~68 GB resident — the 51B-parameter n-gram table lives in ngram_table.bin
  and is memory-mapped from disk, not held in RAM
• ~56 tok/s decode serial, ~60 tok/s with MTP measured on an M5 Max 128 GB
  (custom Metal kernels for the MoE gather, Gated-DeltaNet decode,
  hyper-connections, and sparse attention)
• Vision tower and MTP speculative head included
• Full 262,144-token context
• Base weights are orcarouter's abliteration
  of Qwen/Qwen3.8-Flash-Next —
  refusals removed, everything else intact

The other ways to run this model uncensored on a Mac either don't fit 128 GB
(the original MLX repo is 163 GiB) or run slower (GGUF IQ4_XS ≈ 26 tok/s on
the same hardware; generic Python MLX paths ≈ 27–36 tok/s).

About the base model

This is a 4-bit MLX-Serve pack of
orcarouter/Qwen3.8-Flash-Next-Uncensored
— the full-precision BF16 abliterated (refusal-removed) build of Qwen's
Qwen3.8-Flash-Next. See
orcarouter's card for the full abliteration methodology and evaluation; this
section summarizes it.

⚙️ Requirements

📦 What's in the pack

Built with mlx-serve's own converter
(tests/convert_qwen38_flash_next.py)
from the bf16 uncensored weights:

The architecture's x·(1+w) RMSNorm offset is folded at convert time.

🚀 Run it

1. Get mlx-serve (binary release, or brew install ddalcu/mlx-serve/mlx-serve):

curl -LO https://github.com/ddalcu/mlx-serve/releases/download/v26.8.11-pre-release.1/mlx-serve-bin-macos-arm64.tar.gz
tar xzf mlx-serve-bin-macos-arm64.tar.gz

2. Download this pack (installing hf_transfer first makes this run at full line rate):

pip install -U huggingface_hub hf_transfer
HF_HUB_ENABLE_HF_TRANSFER=1 hf download ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit \
  --local-dir ~/.mlx-serve/models/ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit

3. Chat right in the terminal:

./mlx-serve --model ~/.mlx-serve/models/ARC4NUM/Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit \
  --prompt "Hello!" --max-tokens 512 --temp 0.7

4. Or serve it — OpenAI, Anthropic, and Ollama APIs on one port:

./mlx-serve serve --host 127.0.0.1 --port 11234
# OpenAI:    http://127.0.0.1:11234/v1/chat/completions
# Anthropic: http://127.0.0.1:11234/v1/messages
# Ollama:    http://127.0.0.1:11234/api/chat

Works out of the box with Claude Code, pi, OpenCode, Codex, aider, Open WebUI,
Raycast — anything that speaks one of those wires. mlx-serve launch <agent>
preconfigures and starts your coding agent against it.

MTP speculative decoding (the model's own draft head) is opt-in: pass
--mtp at launch or "enable_mtp": true per request. Fastest on code and
echo-heavy agent loops; a touch slower on freeform prose.

🎛️ Recommended Settings

Qwen's official sampling settings for this model:

Thinking effort defaults to xhigh and this model will happily ruminate;
for agent work, --reasoning-budget 2048 (or the reasoning_effort request
field: low / medium / xhigh) keeps it moving. The model supports
preserve thinking — keeping prior turns' traces in context — which costs
tokens but can help multi-turn accuracy.

📊 Measured performance

M5 Max, 128 GB, this pack, --temp 0 300-token generations:

The Python paths are kernel-bound, not bandwidth-bound — the n-gram gathers,
hyper-connections, and GDN decode run as generic graphs there. mlx-serve's
fused kernels are the whole difference. The author reports ~60 serial / 78 MTP
on an M4 Max for the base-model pack; MTP wiring is still marked partial in
this pre-release, so expect free speed from future engine updates.

Context memory: only 12 of 48 layers are full attention (2 KV heads ×
256 head-dim), so KV cache costs ~24 KB/token — ~2.4 GB at 100k context. Long
agent sessions fit comfortably inside 128 GB.

🧪 Reproduce this pack

python3 tests/convert_qwen38_flash_next.py \
  --src /path/to/Qwen3.8-Flash-Next-Uncensored-bf16 \
  --dst ./Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit
python3 tests/convert_qwen38_flash_next.py --add-vision \
  --src /path/to/Qwen3.8-Flash-Next-Uncensored-bf16 \
  --dst ./Qwen3.8-Flash-Next-Uncensored-MLX-Serve-4bit

(~10 minutes total; streams shard-by-shard, never needs the full 335 GB in RAM.)

⚠️ Disclaimer — read before use

Reproduced from the source model card
and applying equally to this quantized redistribution, which performs format
conversion only.

This model has had its safety alignment substantially removed via
abliteration (orthogonalizing the refusal direction out of the residual
stream). As a direct consequence:

• It will comply with harmful, unethical, offensive, or illegal requests
  that the original Qwen3.8-Flash-Next would refuse. It has no meaningful
  built-in guardrails.
• It is released strictly for legitimate research — interpretability,
  AI-safety and refusal-mechanism study, red-teaming, robustness evaluation,
  and controlled experiments.
• You assume full responsibility and liability for how you use it and for
  everything it generates. Do not deploy it to end users or in production
  without adding your own safety, moderation, and abuse-prevention layers.
• Use must comply with the Apache 2.0 License
  inherited from the base model, and all laws and regulations that apply to you.
• The authors and uploaders accept no liability for any misuse or harm. Its
  outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

By downloading or using this model you acknowledge and accept the above.

Attribution & lineage

This repository changes precision and file format only — no weights were
trained, merged, or otherwise modified beyond quantization. The chain:

License: Qwen Community 1.0,
inherited unchanged from the base model and applying to this derivative. Review
it before commercial use.