license: other
license_name: qwen-community-1.0
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
base_model:

• Qwen/Qwen3.8-Flash-Next-FP8
  pipeline_tag: image-text-to-text
  library_name: mlx
  tags:
• qwen
• qwen3.8
• mlx
• mlx-vlm
• abliterix
• quantized
• mtp
• multimodal
• vision-language
  language:
• multilingual

Qwen3.8-Flash-Next FP8 Abliterixed — MLX 4/6/8-bit

[!IMPORTANT]
This is an unofficial community derivative of Qwen/Qwen3.8-Flash-Next-FP8, produced from the official FP8 checkpoint, processed for reduce refusal-related behavior, and converted to MLX format.

The repository provides MLX affine 4-bit, 6-bit, and 8-bit variants under 4bit/, 6bit/, and 8bit/. All three use group size 64. The 6bit/ variant additionally preserves the source MTP tensors under 6bit/MTP/ as a research/integration sidecar.

This model is not an official Qwen release and is not endorsed by the Qwen Team, Alibaba Cloud, Apple, MLX/MLX-VLM maintainers, or the Abliterix authors.

Model Summary

Qwen3.8-Flash-Next is an experimental preview of the architecture intended to underpin Qwen4. It is a multimodal Mixture-of-Experts model with text, image, and video understanding. Its architecture combines Qwen Sparse Attention, Gated DeltaNet, gated residual connections, n-gram embeddings, and routed experts for long-context and agentic workloads.

This repository preserves the model structure while applying an Abliterix refusal-direction intervention and distributing the result in three MLX quantization variants. After consolidating the original tiny weight shards, the complete repository contains 878 files and approximately 619.24 GB of data.

MLX Quantization

Each directory contains its own complete model configuration and weight index. The default quantization recipe is:

For each variant, 1,401 former sub-100 MiB shards were losslessly repacked into one indexed Safetensors file. The 274 pre-existing large weight shards and config.json were not modified:

These are MLX checkpoints, not GGUF files and not the original Transformers/ModelOpt FP8 layout. Do not load them with llama.cpp, and do not interpret 4bit, 6bit, or 8bit as GGUF quantization names.

Repository Layout

Qwen3.8-Flash-Next-FP8-Abliterixed-MLX/
├── 4bit/                       # MLX affine 4-bit, group size 64
│   ├── config.json
│   ├── model.safetensors.index.json
│   ├── model-consolidated-small.safetensors  # consolidated former tiny shards
│   ├── model-*.safetensors                   # 274 unchanged large shards
│   ├── tokenizer files
│   └── processor / chat-template files
├── 6bit/                       # MLX affine 6-bit, group size 64
│   ├── config.json
│   ├── model.safetensors.index.json
│   ├── model-consolidated-small.safetensors  # consolidated former tiny shards
│   ├── model-*.safetensors                   # 274 unchanged large shards
│   ├── tokenizer files
│   └── processor / chat-template files
├── MTP/
│   ├── model.safetensors.index.json
│   ├── mtp-model-00001-of-00014.safetensors
│   ├── ...
│   ├── mtp-model-00014-of-00014.safetensors
│   └── mtp_manifest.json
└── 8bit/                       # MLX affine 8-bit, group size 64
    ├── config.json
    ├── model.safetensors.index.json
    ├── model-consolidated-small.safetensors  # consolidated former tiny shards
    ├── model-*.safetensors                   # 274 unchanged large shards
    ├── tokenizer files
    └── processor / chat-template files

Do not combine files from different variants. The sharded weight index in each directory refers only to that directory's files.

MTP Sidecar

The 6bit/ directory preserves the source Multi-Token Prediction tensors separately from the main MLX weight index:

• Format: MLX-compatible Safetensors sidecar
• Tensor count: 3,101
• Shards: 14
• Indexed tensor bytes: 2,698,026,496
• Precision: preserved source precision; not converted to the main 6-bit affine recipe

This sidecar is retained for research and future runtime integration. It is not automatically loaded with the main model and is not advertised as a standalone drop-in draft model. Do not pass 6bit/MTP/ to --draft-model unless the chosen runtime explicitly supports this exact Qwen3.8-Flash-Next MTP layout and manifest. Ordinary inference uses only the main files referenced by 6bit/model.safetensors.index.json.

The 4bit/ and 8bit/ variants do not include this separate MTP sidecar.

Loading with MLX

[!WARNING]
This checkpoint uses the experimental qwen4_exp multimodal architecture. Use a recent, compatible MLX-VLM build that explicitly supports qwen4_exp. Runtime support is evolving; verify text generation first, then test image/video input. A generic mlx-lm build that does not include this architecture will not load the checkpoint merely because the weights use MLX format.

MLX runs on Apple silicon. These checkpoints are exceptionally large: the selected directory must fit on disk, and inference needs additional unified-memory headroom beyond the folder size. Systems without sufficient unified memory should use a runtime-supported MoE offloading workflow or a smaller model.

Install the runtime:

python -m pip install -U mlx-vlm huggingface_hub

Download one complete variant (example: 4bit/):

hf download GlobalCybersecurityAlliance/Qwen3.8-Flash-Next-FP8-Abliterixed-MLX \
  --include "4bit/*" \
  --local-dir ./Qwen3.8-Flash-Next-FP8-Abliterixed-MLX

Official Generation Guidance

The official model card states that thinking mode is enabled by default and recommends:

Choosing a Variant

No benchmark claim is made for these specific variants. Measure quality, throughput, prompt-processing speed, and memory use on the target system before deployment.

Intended Use

Appropriate uses include:

• research into model behavior, alignment, refusal, robustness, and quantization;
• authorized evaluation and red-team testing;
• MLX and long-context deployment experiments on compatible Apple silicon;
• defensive cybersecurity education and research in authorized environments;
• evaluation of 4-bit, 6-bit, and 8-bit MLX trade-offs;
• research into compatible MTP integration using the preserved 6-bit sidecar.

Prohibited and High-Risk Use

Do not use this model to facilitate unlawful activity, unauthorized access, credential theft, malware deployment, privacy invasion, harassment, violence, fraud, or other harm. Operators should implement access controls, monitoring, rate limits, content safeguards, and human review appropriate to the deployment context.

Disclaimer

[!CAUTION]
USE AT YOUR OWN RISK. Abliterix processing intentionally changes refusal-related behavior and may weaken safeguards present in the official model. The model may generate inaccurate, unsafe, offensive, biased, unlawful, or otherwise harmful content.

This repository and its files are provided “AS IS” and “AS AVAILABLE,” without warranties or conditions of any kind. Users and deployers are solely responsible for evaluating the model, complying with applicable law and platform policy, obtaining authorization before cybersecurity testing, restricting access, implementing safeguards, and independently verifying outputs.

The maintainers, contributors, quantizers, distributors, the Qwen Team, Alibaba Cloud, Apple, MLX/MLX-VLM maintainers, and the Abliterix authors are not responsible for prompts, outputs, decisions, deployments, damages, losses, claims, liabilities, or consequences arising from use or misuse of this derivative, to the maximum extent permitted by applicable law.

The model must not be treated as professional medical, legal, financial, safety, security, or operational advice.

[!IMPORTANT]
本仓库是 Qwen/Qwen3.8-Flash-Next-FP8 的非官方社区衍生版本，基于官方 FP8 检查点 制作，对拒答相关方向进行处理，并转换为 MLX 格式。

仓库在 4bit/、6bit/ 和 8bit/ 中提供 MLX affine 4-bit、6-bit 与 8-bit 版本，三者的 group size 均为 64。6bit/ 还在 6bit/MTP/ 中保留了源 MTP 张量，供研究和后续运行时集成使用。

本模型并非 Qwen 官方发布，也未获得 Qwen 团队、阿里云、Apple、MLX/MLX-VLM 维护者或 Abliterix 作者的认可或背书。

模型简介

Qwen3.8-Flash-Next 是支持文本、图像和视频理解的多模态混合专家模型。其架构结合 Qwen Sparse Attention、Gated DeltaNet、门控残差、n-gram embedding 与路由专家，面向长上下文和 Agent 工作负载。

本仓库在保留模型结构的基础上进行了 Abliterix 拒答方向干预，并将结果以三种 MLX 量化形式发布。原有零碎权重分片合并后，完整仓库包含 878 个文件，总体积约 619.24 GB。

MLX 量化说明

每个版本原有的 1,401 个小于 100 MiB 的零碎分片均以无损方式重新打包为一个带索引的 Safetensors 文件；原有 274 个大权重分片和 config.json 均未修改：

这些文件是 MLX 检查点，不是 GGUF，也不是原始 Transformers/ModelOpt FP8 布局。不要使用 llama.cpp 加载，也不要将 4bit、6bit 或 8bit 理解为 GGUF 量化名称。

文件结构

Qwen3.8-Flash-Next-FP8-Abliterixed-MLX/
├── 4bit/                       # MLX affine 4-bit，group size 64
│   ├── model-consolidated-small.safetensors  # 原零碎分片合并文件
│   └── model-*.safetensors                   # 274 个未改动的大分片
├── 6bit/                       # MLX affine 6-bit，group size 64
│   ├── model-consolidated-small.safetensors  # 原零碎分片合并文件
│   ├── model-*.safetensors                   # 274 个未改动的大分片
│
├── 8bit/                       # MLX affine 8-bit，group size 64
│   ├── model-consolidated-small.safetensors  # 原零碎分片合并文件
│   └── model-*.safetensors                   # 274 个未改动的大分片
├── MTP/                                  # 14 个 MTP 分片及其索引和 manifest

每个版本目录都包含独立的 config.json、model.safetensors.index.json、模型分片、tokenizer、processor 和 chat template 文件。不要混合不同目录中的文件；每个权重索引只对应同目录中的分片。

MTP Sidecar

6bit/ 将 Multi-Token Prediction 张量与主模型权重索引分开保存：

• 格式： MLX 兼容 Safetensors sidecar
• 张量数： 3,101
• 分片数： 14
• 索引张量字节数： 2,698,026,496
• 精度： 保留源精度，不采用主模型的 6-bit affine 方案

该 sidecar 用于研究和后续运行时集成，不会随主模型自动加载，也不应被视为可直接使用的独立 draft model。除非所用运行时明确支持这一特定的 Qwen3.8-Flash-Next MTP 布局及 manifest，否则不要把 6bit/MTP/ 传给 --draft-model。普通推理只使用 6bit/model.safetensors.index.json 引用的主模型文件。

4bit/ 和 8bit/ 不包含这一独立 MTP sidecar。

使用 MLX 加载

[!WARNING]
本检查点使用实验性的 qwen4_exp 多模态架构。请使用明确支持 qwen4_exp 的新版兼容 MLX-VLM。相关运行时支持仍在演进，建议先验证文本生成，再测试图像和视频输入。仅仅因为权重采用 MLX 格式，并不表示缺少该架构实现的通用 mlx-lm 版本能够直接加载。

MLX 运行于 Apple silicon。三个检查点的体积都非常大：磁盘必须容纳完整所选目录，推理时还需要高于目录体积的统一内存余量。统一内存不足时，应使用运行时支持的 MoE offloading 工作流或选择更小的模型。

安装运行时：

python -m pip install -U mlx-vlm huggingface_hub

下载一个完整版本（以 4bit/ 为例）：

hf download GlobalCybersecurityAlliance/Qwen3.8-Flash-Next-FP8-Abliterixed-MLX \
  --include "4bit/*" \
  --local-dir ./Qwen3.8-Flash-Next-FP8-Abliterixed-MLX

官方生成参数建议

官方模型卡说明 thinking mode 默认开启，并推荐：

版本选择

本仓库不对这些具体版本作基准性能声明。部署前请在目标设备上测试质量、吞吐量、prompt 处理速度和内存占用。

预期用途

适合的用途包括：模型行为、对齐、拒答、鲁棒性和量化研究；经授权的评估与红队测试；兼容 Apple silicon 上的 MLX 与长上下文部署实验；合法授权环境中的防御性网络安全教育与研究；4-bit、6-bit、8-bit MLX 版本的质量和资源权衡测试；使用保留的 6-bit sidecar 开展兼容 MTP 集成研究。

禁止及高风险用途

不得使用本模型实施或协助违法活动、未经授权的系统访问、凭据窃取、恶意软件投放、侵犯隐私、骚扰、暴力、欺诈或其他伤害行为。部署者应根据应用场景实施访问控制、审计监控、速率限制、内容保护和人工复核。

免责声明

[!CAUTION]
使用者自行承担全部风险。 Abliterix 会主动改变拒答相关行为，并可能削弱官方模型的部分安全保护。模型可能生成错误、不安全、冒犯性、偏见性、违法或其他有害内容。

本仓库及其文件均按**“现状”和“可用状态”**提供，不作任何明示或默示保证。使用者和部署者必须自行评估模型、遵守适用法律法规与平台规则、在网络安全测试前取得授权、限制访问、实施必要保护措施，并独立核验模型输出。

在适用法律允许的最大范围内，维护者、贡献者、量化者、分发者、Qwen 团队、阿里云、Apple、MLX/MLX-VLM 维护者以及 Abliterix 作者，不对因使用或误用本衍生模型而产生的提示词、输出、决策、部署、损害、损失、索赔、责任或后果负责。

本模型不得被视为专业医疗、法律、金融、安全、网络安全或运营建议。