license: other
license_name: qwen-community-license-1.0
license_link: https://huggingface.co/Qwen/Qwen3.8-Flash-Next/blob/main/LICENSE
base_model:

• Qwen/Qwen3.8-Flash-Next
  pipeline_tag: image-text-to-text
  library_name: llama.cpp
  tags:
  • qwen
  • qwen3.8
  • gguf
  • abliterix
  • quantized
  • mtp
  • speculative-decoding
  • multimodal
  • vision-language
  • llama.cpp
    language:
  • multilingual

Qwen3.8-Flash-Next FP8 Abliterixed — GGUF + MTP Collection

[!IMPORTANT]
This is an unofficial community derivative of Qwen/Qwen3.8-Flash-Next-FP8, produced from the official FP8 checkpoint, processed with Abliterix to reduce refusal-related behavior, and converted to GGUF.

The repository provides target-model GGUFs in Q8_0, Q6_K, and Q4_K_M, plus a matching standalone MTP GGUF sidecar in each quantization directory for optional speculative decoding. These GGUF labels describe the distributed files' precision; they are distinct from the mixed ModelOpt layout of the intermediate source checkpoint.

This model is not an official Qwen release and is not endorsed by the Qwen Team, Alibaba Cloud, NVIDIA, or the Abliterix authors.

Model Summary

Qwen3.8-Flash-Next is a multimodal Mixture-of-Experts model with text, image, and video understanding. The official architecture combines Qwen Sparse Attention, Gated DeltaNet, gated residual connections, n-gram embeddings, and routed experts for long-context and agentic workloads.

This repository preserves that architecture while applying an Abliterix refusal-direction intervention and distributing the result as three GGUF quantizations. The target weights and MTP sidecars are organized under Q4_KM/, Q6_K/, and Q8/. The repository page reports an aggregate size of approximately 485 GB.

Files

The repository is organized into three download directories. The folder labels visible in the repository and their actual GGUF schemes are:

Large GGUFs may be split into multiple files. Inspect the selected directory for its exact shard count; the structure follows this pattern:

[!NOTE]
The repository folders are named Q4_KM, Q6_K, and Q8, while the actual GGUF schemes are Q4_K_M, Q6_K, and Q8_0. Folder names and quantization names are therefore not identical in two cases.

MTP Speculative Decoding

Qwen3.8-Flash-Next includes native Multi-Token Prediction components. In this repository they are published as standalone GGUF sidecars so the target model and MTP draft can be loaded separately. The draft proposes future tokens; the target model verifies them and rejects mismatches.

MTP is optional. Ordinary inference requires only the target-model shards. Enabling MTP additionally requires the matching sidecar and a llama.cpp build supporting both the qwen4exp architecture and draft-mtp. Actual gains depend on acceptance rate, hardware, backend, offload layout, prompt length, batch size, and speculative depth.

Always pair a target with the MTP sidecar from the same directory. Do not mix Q4_K_M, Q6_K, and Q8_0 sidecars. Compare MTP against ordinary decoding on the same workload; a larger draft depth can be slower when acceptance is low.

[!WARNING]
Use a recent llama.cpp build that explicitly supports the qwen4exp/Qwen3.8-Flash-Next architecture. Support landed recently and backend-specific issues may remain. Test ordinary decoding before enabling MTP, and verify the exact build on the target hardware.

Intended Use

Appropriate uses include:

• research into model behavior, alignment, refusal, robustness, and quantization;
• authorized evaluation and red-team testing;
• GGUF and long-context deployment experiments;
• defensive cybersecurity education and research in authorized environments;
• evaluation of Q8_0, Q6_K, and Q4_K_M tradeoffs;
• benchmarking MTP speculative decoding and draft acceptance on compatible llama.cpp backends.

Prohibited and High-Risk Use

Do not use this model to facilitate unlawful activity, unauthorized access, credential theft, malware deployment, privacy invasion, harassment, violence, fraud, or other harm. Operators should implement access controls, monitoring, rate limits, content safeguards, and human review appropriate to the deployment context.

Disclaimer

[!CAUTION]
USE AT YOUR OWN RISK. Abliterix processing intentionally changes refusal-related behavior and may weaken safeguards present in the official model. The model may generate inaccurate, unsafe, offensive, biased, unlawful, or otherwise harmful content.

This repository and its files are provided “AS IS” and “AS AVAILABLE,” without warranties or conditions of any kind. Users and deployers are solely responsible for evaluating the model, complying with applicable law and platform policy, obtaining authorization before cybersecurity testing, restricting access, implementing safeguards, and independently verifying outputs.

The maintainers, contributors, quantizers, distributors, the Qwen Team, Alibaba Cloud, NVIDIA, and the Abliterix authors are not responsible for prompts, outputs, decisions, deployments, damages, losses, claims, liabilities, or consequences arising from use or misuse of this derivative, to the maximum extent permitted by applicable law.

The model must not be treated as professional medical, legal, financial, safety, security, or operational advice.

[!IMPORTANT]
本仓库是 Qwen/Qwen3.8-Flash-Next-FP8 的非官方社区衍生版本，基于官方 FP8 检查点制作，使用 Abliterix 对拒答相关方向进行处理，并转换为 GGUF。

仓库提供 Q8_0、Q6_K 和 Q4_K_M 三种目标模型 GGUF，并在每个量化目录中提供配套的独立 MTP GGUF sidecar，用于可选的推测解码。这里的 GGUF 量化标签描述最终分发文件的精度，与中间源检查点的 ModelOpt 混合精度布局并不是同一概念。

模型简介

Qwen3.8-Flash-Next 是支持文本、图像和视频理解的多模态混合专家模型。官方架构结合 Qwen Sparse Attention、Gated DeltaNet、门控残差、n-gram embedding 与路由专家，面向长上下文和 Agent 工作负载。

本仓库在保留该架构的基础上进行了 Abliterix 拒答方向干预，并将结果以三种 GGUF 量化形式发布。目标模型权重和 MTP sidecar 分别位于 Q4_KM/、Q6_K/ 和 Q8/。仓库页面显示总体积约为 485 GB。

与官方模型的关系

官方 Qwen 模型卡中的评测结果仅代表未经修改的官方模型，不能视为本 Abliterix 衍生版及选择性量化版的实测结果。本仓库不保证完整保持官方模型的能力、评测分数、安全特性或数值行为。

文件说明

仓库包含三个下载目录。目录名称与实际 GGUF 量化方案的对应关系如下：Q8/ 对应 Q8_0，Q6_K/ 对应 Q6_K，Q4_KM/ 对应 Q4_K_M。每个目录都包含目标模型 GGUF 分片及相应的 MTP GGUF sidecar。

[!NOTE]
仓库目录名为 Q4_KM、Q6_K 和 Q8，实际 GGUF 量化方案则为 Q4_K_M、Q6_K 和 Q8_0，因此其中两个目录名与量化名并不完全相同。

MTP 推测解码

Qwen3.8-Flash-Next 包含原生 Multi-Token Prediction 组件。本仓库将其发布为独立 GGUF sidecar，使目标模型和 MTP 草稿可以分别加载。草稿模型预先提出后续 token，再由目标模型验证并拒绝不匹配的候选。

MTP 是可选优化。普通推理只需要目标模型分片；启用 MTP 还需要匹配的 sidecar，以及同时支持 qwen4exp 架构和 draft-mtp 的 llama.cpp 版本。实际收益取决于候选接受率、硬件、后端、卸载方式、提示词长度、批大小和推测深度。

务必将目标模型与同一目录中的 MTP sidecar 配对，不要混用 Q4_K_M、Q6_K 和 Q8_0 sidecar。请在相同工作负载下对比 MTP 与普通解码；候选接受率较低时，更大的草稿深度反而可能更慢。

[!WARNING]
请使用明确支持 qwen4exp/Qwen3.8-Flash-Next 架构的新版 llama.cpp。相关支持较新，特定后端仍可能存在问题。请先验证普通解码，再启用 MTP，并在目标硬件上测试实际构建版本。

预期用途

适合的用途包括：模型行为、对齐、拒答、鲁棒性和量化研究；经授权的评估与红队测试；GGUF 及长上下文部署实验；合法授权环境中的防御性网络安全教育与研究；Q8_0、Q6_K 与 Q4_K_M 的质量和资源权衡测试；兼容 llama.cpp 后端上的 MTP 推测解码与草稿接受率评测。

禁止及高风险用途

不得使用本模型实施或协助违法活动、未经授权的系统访问、凭据窃取、恶意软件投放、侵犯隐私、骚扰、暴力、欺诈或其他伤害行为。部署者应根据应用场景实施访问控制、审计监控、速率限制、内容保护和人工复核。

免责声明

[!CAUTION]
使用者自行承担全部风险。 Abliterix 会主动改变拒答相关行为，并可能削弱官方模型的部分安全保护。模型可能生成错误、不安全、冒犯性、偏见性、违法或其他有害内容。

本仓库及其文件均按**“现状”和“可用状态”**提供，不作任何明示或默示保证。使用者和部署者必须自行评估模型、遵守适用法律法规与平台规则、在网络安全测试前取得授权、限制访问、实施必要保护措施，并独立核验模型输出。