license: apache-2.0
base_model: Qwen/Qwen3.8-27B
pipeline_tag: image-text-to-text
library_name: transformers
tags:

• qwen
• qwen3.8
• abliterix
• compressed-tensors
• safetensors
• vllm
• quantized
• int8
• int4
• fp8
• fp4
• bf16
• multimodal
• vision-language
  language:
• multilingual

Qwen3.8-27B Abliterated — BF16, INT8, INT4, FP8 and FP4 Collection

[!IMPORTANT]
This is an unofficial community derivative of Qwen/Qwen3.8-27B. It has been processed with Abliterix to reduce selected refusal-related behavior and is distributed as an unquantized BF16 checkpoint plus four compressed-tensors quantizations: INT8, INT4, FP8, and FP4.

This is not an official Qwen release and is not endorsed by the Qwen Team, Alibaba Cloud, NVIDIA, the vLLM/compressed-tensors/LLM Compressor maintainers, or the Abliterix authors.

本仓库是 Qwen/Qwen3.8-27B 的非官方社区衍生版本。模型经过 Abliterix 处理，以降低部分拒答相关行为，并提供未量化的 BF16 检查点，以及采用 compressed-tensors 格式的 INT8、INT4、FP8 和 FP4 四种量化版本。

本模型并非 Qwen 官方发布，也不代表 Qwen 团队、阿里云、NVIDIA、vLLM/compressed-tensors/LLM Compressor 维护者或 Abliterix 作者的立场或认可。

Model Summary / 模型简介

Qwen3.8-27B is a dense, approximately 27-billion-parameter native vision-language model supporting text, image, and video understanding. This repository preserves the derivative architecture while applying an Abliterix refusal-direction intervention. It provides the resulting BF16 derivative as the highest-fidelity reference build, together with four post-training quantizations for deployment-oriented experimentation.

Qwen3.8-27B 是一个约 270 亿参数的原生视觉语言模型，支持文本、图像和视频理解。本仓库在保留衍生模型架构的基础上，使用 Abliterix 对拒答相关方向进行干预，并将得到的 BF16 衍生模型作为最高保真参考版本，同时提供四种面向部署实验的训练后量化版本。

Relationship to the official model / 与官方模型的关系

• Official base model / 官方基础模型： Qwen/Qwen3.8-27B
• Behavioral intervention / 行为干预： Abliterix refusal-direction intervention
• Storage format / 存储格式： sharded Safetensors; quantized variants include compressed-tensors metadata
• Available variants / 可用版本： BF16, INT8, INT4, FP8, FP4
• Repository / 仓库： GlobalCybersecurityAlliance/Qwen3.8-27B-Abliterated
• Maintainer / 维护者： Global Cybersecurity Alliance (community distribution)

The official Qwen benchmark results describe the unmodified base model only. They must not be interpreted as results for this Abliterix-processed or quantized derivative. No claim is made that this derivative preserves every capability or score of the official model.

官方 Qwen 模型卡中的基准测试结果仅代表未经修改的基础模型，不能视为本 Abliterix 衍生版或量化版的实测成绩。本仓库不保证衍生模型完整保持官方模型的全部能力或分数。

Available Variants / 可用版本

The BF16 folder contains the unquantized Abliterix-processed derivative in native BFloat16 Safetensors. The other four variants use compressed-tensors 0.18-compatible metadata. In the quantized variants, the vision encoder, multimodal projector, token embeddings, lm_head, and selected linear-attention convolution modules were excluded from quantization to preserve compatibility and stability. Consequently, actual repository sizes are larger than a simple parameter-count × bit-width estimate.

BF16 目录包含经 Abliterix 处理后、以原生 BFloat16 Safetensors 保存的未量化衍生模型。其余四个版本均包含兼容 compressed-tensors 0.18 的量化元数据。为了兼容性和稳定性，量化版本中的视觉编码器、多模态投影器、词嵌入、lm_head 以及部分线性注意力卷积模块未量化。因此，实际文件大小会高于“参数量 × 位宽”的简单理论估算。

Each folder is a self-contained model directory containing configuration files, tokenizer and processor assets, chat template, 55 weight shards, and a weight index. The four quantized folders additionally contain quantization metadata.

每个目录都是独立、完整的模型目录，包含配置文件、分词器和处理器文件、聊天模板、55 个权重分片及权重索引；四个量化目录还包含相应的量化元数据。

QUANTIZATION_COMPLETE.json, when present, is build-verification metadata and is not required for inference.

Precision and Quantization Notes / 精度与量化说明

BF16

BF16 is the unquantized Abliterix-processed derivative and serves as the highest-fidelity reference checkpoint in this collection. It avoids additional quantization error and is the preferred source for evaluation, further conversion, fine-tuning experiments, or deployments where sufficient accelerator memory is available. Runtime memory will exceed the approximately 51GB weight size because additional space is required for the vision encoder execution, KV cache, activations, CUDA/runtime buffers, batching, and context processing.

BF16 是经 Abliterix 处理后的未量化衍生模型，也是本集合中保真度最高的参考检查点。它不会引入额外量化误差，适合用于效果评估、进一步格式转换、微调实验，或在加速器内存充足时直接部署。实际运行内存会高于约 51GB 的权重体积，因为视觉编码器执行、KV Cache、激活值、CUDA/运行时缓冲区、批处理和上下文处理都需要额外空间。

INT8

INT8 uses static per-channel symmetric 8-bit integer weights and dynamic per-token symmetric 8-bit integer activations. It generally requires more memory than the 4-bit variants but offers a less aggressive integer quantization.

INT8 使用静态逐通道对称 8 位整数权重及动态逐 token 对称 8 位整数激活。其内存需求通常高于 4 位版本，但整数压缩程度相对保守。

INT4

INT4 is a W4A16 weight-only scheme: weights are packed as symmetric 4-bit integers with group size 128, while activations remain at 16-bit precision. This provides the smallest integer-weight variant in this collection.

INT4 使用 W4A16 仅权重量化：权重以 group size 128 的对称 4 位整数打包，激活保持 16 位精度。这是本集合中体积最小的整数权重版本。

FP8

FP8 uses per-channel 8-bit floating-point weights and dynamic per-token 8-bit floating-point activations. Efficient execution depends on suitable accelerator and runtime support.

FP8 使用逐通道 8 位浮点权重和动态逐 token 8 位浮点激活。能否高效运行取决于加速器和推理框架是否提供相应支持。

FP4

FP4 uses the NVFP4A16 scheme: 4-bit floating-point weights with group size 16 and FP8 E4M3 local scales, while activations remain at 16-bit precision. Native acceleration is hardware-dependent; unsupported platforms may fail to load the model or fall back to a slower path if the runtime provides one.

FP4 使用 NVFP4A16：权重为 group size 16 的 4 位浮点格式，并使用 FP8 E4M3 局部缩放因子，激活保持 16 位精度。原生加速依赖硬件；不支持的平台可能无法加载，或在运行时提供回退路径时以较慢方式执行。

What Abliterix Changes / Abliterix 修改说明

Abliterix identifies and modifies activation directions associated with refusal behavior. The intended effect is to reduce excessive refusals on selected evaluations. This is a weight-level behavioral intervention rather than a system-prompt override.

Abliterix 用于识别并修改与拒答行为相关的激活方向，目标是在特定评测中减少过度拒答。这属于权重层面的行为干预，并非简单覆盖系统提示词。

Important limitations / 重要限制：

• Reduced refusal behavior is not guaranteed for every prompt, language, chat template, or inference engine.
• The model may still refuse requests because refusal behavior can be distributed across multiple layers and mechanisms.
• Abliterix processing may affect tone, calibration, reasoning quality, factual accuracy, safety behavior, or instruction following.
• Quantization may introduce additional quality differences compared with the BF16 derivative.
• The internal label best15 is a build-selection label, not a standardized or independently reproduced benchmark score.
• “Abliterated” or reduced refusal does not mean unrestricted capability, guaranteed compliance, correctness, or safety.
• 模型不保证对所有提示词、语言、聊天模板或推理框架都降低拒答。
• 拒答行为可能分布于多个层和机制，因此模型仍可能拒绝部分请求。
• Abliterix 处理可能影响语气、置信度校准、推理质量、事实准确性、安全行为或指令遵循能力。
• 量化可能进一步造成与 BF16 衍生模型不同的质量变化。
• 内部标签 best15 仅用于构建版本筛选，不是标准化或独立复现的评测分数。
• “Abliterated”或拒答率降低不代表模型具备无限能力，也不保证服从、正确或安全。

Runtime and Hardware Compatibility / 运行时与硬件兼容性

The BF16 checkpoint is standard unquantized sharded Safetensors and does not use a quantization backend. The INT8, INT4, FP8, and FP4 checkpoints use compressed-tensors, not GGUF, MLX, GPTQ, AWQ, or bitsandbytes format. A recent vLLM build is the recommended starting point because it can load BF16 directly and read quantization metadata from each quantized variant's config.json.

BF16 检查点是标准的未量化分片 Safetensors，不依赖量化后端。INT8、INT4、FP8 和 FP4 检查点使用 compressed-tensors，并非 GGUF、MLX、GPTQ、AWQ 或 bitsandbytes 格式。建议优先使用最新版 vLLM：BF16 可直接加载，量化版本则由 vLLM 读取各自 config.json 中的量化元数据。

[!WARNING]
Quantized-kernel support varies by vLLM release, GPU architecture, CUDA/ROCm version, operating system, and model architecture. A checkpoint being structurally valid does not guarantee that every backend can execute it. Verify compatibility with the exact hardware and software stack before deployment.

不同 vLLM 版本、GPU 架构、CUDA/ROCm 版本、操作系统和模型架构支持的量化内核不同。检查点结构有效并不代表所有后端都能运行。部署前必须在目标软硬件环境中验证兼容性。

FP8 and NVFP4 generally benefit most from GPUs with native support for the corresponding number formats. INT4 and INT8 also require compatible kernels. On unsupported hardware, use another variant or the BF16 derivative rather than assuming transparent compatibility. BF16 itself requires hardware and software with BFloat16 support and substantially more memory than the quantized variants.

FP8 和 NVFP4 通常在原生支持相应数值格式的 GPU 上收益最大；INT4 和 INT8 同样需要兼容内核。不支持时，应改用其他量化版本或 BF16 衍生版，不应假设推理框架能够自动兼容。BF16 本身也要求软硬件支持 BFloat16，并且需要显著高于量化版本的内存。

Download a Variant / 下载指定版本

Because the variants are stored in subfolders, download the required folder first. The example below downloads BF16.

由于五个版本存放于不同子目录，请先下载所需版本。以下示例下载 BF16。

python -m pip install -U huggingface_hub

hf download GlobalCybersecurityAlliance/Qwen3.8-27B-Abliterated \
  --include "BF16/*" \
  --local-dir ./Qwen3.8-27B-Abliterated

The resulting local model path is:

./Qwen3.8-27B-Abliterated/BF16

Replace BF16 with INT8, INT4, FP8, or FP4 in both places to download another variant.

Quick Start with vLLM / 使用 vLLM

Install a recent vLLM build appropriate for the target CUDA/ROCm environment. Check the official vLLM installation documentation rather than blindly reusing an installation command from another machine.

请根据目标 CUDA/ROCm 环境安装合适的最新版 vLLM。建议查阅 vLLM 官方安装文档，不要直接照搬其他机器的安装命令。

Start an OpenAI-compatible server / 启动兼容 OpenAI API 的服务

vllm serve ./Qwen3.8-27B-Abliterated/BF16 \
  --served-model-name Qwen3.8-27B-Abliterated-BF16 \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --host 0.0.0.0 \
  --port 8000

For multi-GPU deployment, add an appropriate tensor-parallel setting, for example:

--tensor-parallel-size 2

Do not assume that the example context length or tensor-parallel size is suitable for every system. Adjust context length, GPU memory utilization, batching, and parallelism according to available memory and runtime validation.

不要假设示例中的上下文长度或张量并行数适合所有系统。请根据可用显存和实测结果调整上下文长度、GPU 内存利用率、批处理和并行参数。

Text request / 文本请求

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen3.8-27B-Abliterated-BF16",
    "messages": [
      {"role": "user", "content": "Explain the difference between symmetric and asymmetric encryption."}
    ],
    "temperature": 0.6,
    "max_tokens": 512
  }'

For image or video input, follow the multimodal request format supported by the installed vLLM release and confirm that the Qwen architecture is supported. Multimodal payload formats can change between releases.

对于图像或视频输入，请遵循当前安装版本 vLLM 支持的多模态请求格式，并确认其支持本 Qwen 架构。不同版本的多模态请求格式可能变化。

Choosing a Variant / 如何选择版本

• Choose BF16 for the highest-fidelity reference, evaluation, further conversion, or deployment when sufficient memory is available.
• Choose INT8 for a less aggressive integer quantization when compatible W8A8 kernels are available.
• Choose INT4 to reduce weight memory and storage while keeping 16-bit activations.
• Choose FP8 when the target accelerator and runtime provide efficient FP8 W8A8 execution.
• Choose FP4 when NVFP4 support is available and minimizing floating-point weight storage is the priority.
• 追求最高保真度、进行效果评估或进一步格式转换，并且内存充足时选择 BF16。
• 具备兼容 W8A8 内核并希望采用相对保守的整数量化时选择 INT8。
• 希望降低权重内存和存储，同时保持 16 位激活时选择 INT4。
• 目标加速器及推理框架能够高效执行 FP8 W8A8 时选择 FP8。
• 具备 NVFP4 支持且优先降低浮点权重体积时选择 FP4。

There is no universally “best” version. Benchmark accuracy, latency, throughput, peak memory, multimodal behavior, and refusal rate on the actual deployment stack.

不存在适用于所有环境的“最佳”版本。请在实际部署环境中测试准确率、延迟、吞吐量、峰值内存、多模态行为和拒答率。

Quantization level does not directly determine refusal rate. Sampling parameters such as temperature and top-p affect output variability but do not reproduce or replace the weight-level Abliterix intervention.

量化位宽并不直接决定拒答率。temperature、top-p 等采样参数会改变输出随机性，但不能复现或替代 Abliterix 的权重层干预。

Intended Use / 预期用途

Appropriate uses include:

• research into model behavior, alignment, refusal, robustness, and quantization;
• authorized evaluation and red-team testing;
• local or controlled vLLM deployment experiments;
• defensive cybersecurity education and research in authorized environments;
• evaluation of multimodal behavior on text, image, and video inputs.

适合的用途包括：模型行为、对齐、拒答、鲁棒性与量化研究；经授权的评估和红队测试；本地或受控环境中的 vLLM 部署实验；合法授权环境中的防御性网络安全教育与研究；文本、图像和视频多模态行为评估。

Prohibited and High-Risk Use / 禁止及高风险用途

Do not use this model to facilitate unlawful activity, unauthorized access, credential theft, malware deployment, privacy invasion, harassment, violence, fraud, exploitation, or other harm. Operators must implement access controls, monitoring, rate limits, content safeguards, and human review appropriate to the deployment context.

不得使用本模型实施或协助违法活动、未经授权的系统访问、凭据窃取、恶意软件投放、侵犯隐私、骚扰、暴力、欺诈、剥削或其他伤害行为。部署者必须根据应用场景实施访问控制、审计监控、速率限制、内容保护和人工复核。

Disclaimer / 免责声明

[!CAUTION]
USE AT YOUR OWN RISK. Abliterix processing intentionally changes refusal-related behavior and may weaken safeguards present in the official model. The model may generate inaccurate, unsafe, offensive, biased, unlawful, insecure, or otherwise harmful content and may follow malicious instructions more readily than the official base model.

This repository and all included files are provided “AS IS” and “AS AVAILABLE,” without warranties or conditions of any kind, express or implied, including but not limited to warranties of merchantability, fitness for a particular purpose, accuracy, reliability, non-infringement, security, safety, compatibility, performance, or uninterrupted availability.

The maintainers, contributors, quantizers, converters, distributors, the Qwen Team, Alibaba Cloud, NVIDIA, vLLM/compressed-tensors/LLM Compressor maintainers, and the Abliterix authors are not responsible for prompts, outputs, decisions, deployments, damages, losses, security incidents, claims, liabilities, or consequences arising from use or misuse of this derivative, to the maximum extent permitted by applicable law. Mention of any organization, product, or software project does not imply endorsement, sponsorship, or involvement.

Users and deployers are solely responsible for:

1. evaluating the model and every available variant before deployment;
2. verifying compatibility with the exact hardware and software stack;
3. complying with applicable laws, regulations, licenses, platform policies, and third-party rights;
4. obtaining explicit authorization before any cybersecurity testing;
5. preventing access by unauthorized or unsuitable users;
6. implementing safeguards appropriate to the use case;
7. independently verifying model outputs before relying on them;
8. protecting personal, confidential, proprietary, and security-sensitive data.

The model must not be treated as professional medical, legal, financial, safety, security, cybersecurity, or operational advice. Do not rely on it for decisions where errors could cause injury, rights violations, financial loss, system compromise, or other material harm.

[!CAUTION]
使用者自行承担全部风险。 Abliterix 处理会主动改变拒答相关行为，可能削弱官方模型原有的部分安全保护。模型可能生成错误、不安全、冒犯性、偏见性、违法、不可靠或其他有害内容，也可能比官方基础模型更容易遵循恶意指令。

本仓库及其中所有文件均按**“现状”和“可用状态”**提供，不作任何明示或默示保证，包括但不限于适销性、特定用途适用性、准确性、可靠性、不侵权性、安全性、无害性、兼容性、性能或持续可用性保证。

在适用法律允许的最大范围内，维护者、贡献者、量化者、转换者、分发者、Qwen 团队、阿里云、NVIDIA、vLLM/compressed-tensors/LLM Compressor 维护者以及 Abliterix 作者，均不对因使用或误用本衍生模型产生的提示词、输出、决策、部署、损害、损失、安全事件、索赔、责任或后果承担责任。文中提及任何组织、产品或软件项目，均不代表其认可、赞助或参与本模型。

使用者和部署者应自行负责：

1. 在部署前充分评估模型及每个可用版本；
2. 验证目标软硬件环境的兼容性；
3. 遵守适用法律法规、许可证、平台规则和第三方权利；
4. 在开展任何网络安全测试前取得明确授权；
5. 防止未经授权或不适合的用户访问；
6. 根据实际用途实施必要的安全保护措施；
7. 在依赖任何模型输出前进行独立核验；
8. 保护个人、机密、专有及安全敏感数据。

本模型不得被视为专业的医疗、法律、金融、安全、网络安全或运营建议。对于错误可能导致人身伤害、权利侵害、经济损失、系统失陷或其他重大损害的决策，不得直接依赖模型输出。

License and Attribution / 许可证与署名

The official base-model repository identifies its license as Apache-2.0. Users must independently review and comply with the base-model license and any terms applicable to dependencies, compressed-tensors, vLLM, inference software, datasets, inputs, outputs, and the intended use.

This derivative does not transfer ownership of the original model, trademarks, documentation, software, or third-party materials. “Qwen,” related marks, and official documentation remain the property of their respective owners.

官方基础模型仓库将许可证标注为 Apache-2.0。使用者必须自行查阅并遵守基础模型、依赖项、compressed-tensors、vLLM、推理软件、数据集、输入、输出及实际用途适用的许可证和相关条款。

本衍生版本不转移原始模型、商标、文档、软件或任何第三方材料的所有权。“Qwen”及相关标识和官方资料仍归各自权利人所有。

References / 参考资料

• Official base model: Qwen/Qwen3.8-27B
• Abliterix
• vLLM
• compressed-tensors
• LLM Compressor