license: apache-2.0
base_model: Qwen/Qwen3.8-27B
language:
- en
- zh
pipeline_tag: text-generation
tags: - qwen3.8
- nvfp4
- mixed-precision
- modelopt
- sglang
- mtp
- multimodal
- safetensors
- sft
- simpo
- efficient-thinking
Qwen3.8-27B EfficientThink — NVFP4 + BF16 MTP
BF16 / FP8 main model · GGUF variants
Research disclaimer: This experimental release is provided solely to study the technical feasibility and behavioral effects of refusal-tendency dissolution. It is not a comprehensive safety conclusion, an endorsement of unrestricted use, or professional advice. Users are responsible for lawful and appropriate use and for independently verifying model outputs.
The figure compares this model's fast and mixed-precision variants, not the official base versus a post-trained model. GPQA reasoning statistics cover 186 / 187 returned requests; accuracy retains all 198 questions. MMLU and LCB reasoning statistics cover 500 / 100 questions. This page retains the frozen C24 capability results.
Formal capability results
| Variant | GPQA 198 | MMLU 500 | LCB 100 |
|---|---|---|---|
| NVFP4 Fast | 158/198 (79.80%) | 430/500 (86.00%) | 74/100 (74.00%) |
| NVFP4 Mixed Precision | 168/198 (84.85%) | 449/500 (89.80%) | 74/100 (74.00%) |
Measured on one RTX PRO 6000 96GB, SGLang, official BF16 MTP, C24, xhigh, a 32,768-token output budget, and a 1,800-second request timeout. All 12 / 11 GPQA timeouts remain in the failure denominator; anomalous questions were not dropped. Mixed precision answers 10 more GPQA and 19 more MMLU questions correctly, with the same LCB total. Results from different runtime formats and decoders are not controlled measurements of quantization loss.
Mixed precision versus fast: scores and reasoning cost
| Metric | W4A4 Fast | W4A4 + W8A8 Mixed | Change |
|---|---|---|---|
| GPQA score | 158/198 (79.80%) | 168/198 (84.85%) | +10 correct / +5.05pp |
| GPQA mean reasoning | 9,123 | 8,400 | -7.9% |
| GPQA P50 / P90 | 4,928.5 / 24,893.0 | 4,278.0 / 24,402.6 | — |
| MMLU score | 430/500 (86.00%) | 449/500 (89.80%) | +19 correct / +3.80pp |
| MMLU mean reasoning | 963 | 848 | -11.9% |
| MMLU P50 / P90 | 225.0 / 2,285.5 | 216.5 / 1,668.9 | — |
| LCB score | 74/100 (74.00%) | 74/100 (74.00%) | Unchanged |
| LCB mean reasoning | 14,602 | 13,647 | -6.5% |
| LCB P50 / P90 | 9,626.5 / 32,769.0 | 7,441.5 / 32,769.0 | — |
Under this C24 protocol, mixed precision answers 10 more GPQA and 19 more MMLU questions correctly, while LCB accuracy is unchanged; mean reasoning tokens are lower in all three suites. This compares quantized variants, not the official base against post-training. GPQA reasoning statistics cover 186 / 187 returned requests, while accuracy retains all 198 questions. MMLU/LCB means cover 500/100 questions; percentage changes use unrounded means.
MTP concurrency, speed and acceptance
This is the existing 256-token fixed-length short-output sweep: one wave per cell, 53 requests per variant. Aggregate throughput is neither per-user speed nor sustained 32K reasoning performance. The C16 cell is a completed short-output speed measurement, not a C16 full capability score.
Fast
| Concurrency | Aggregate tok/s | MTP acceptance | Accepted draft / verification | Completion / verification |
|---|---|---|---|---|
| C1 | 82 | 75.64% | 2.269 | 3.282 |
| C4 | 318 | 76.56% | 2.297 | 3.303 |
| C8 | 688 | 74.49% | 2.235 | 3.225 |
| C16 | 1,196 | 74.25% | 2.227 | 3.223 |
| C24 | 1,684 | 73.17% | 2.195 | 3.197 |
Mixed precision
| Concurrency | Aggregate tok/s | MTP acceptance | Accepted draft / verification | Completion / verification |
|---|---|---|---|---|
| C1 | 102 | 72.92% | 2.188 | 3.200 |
| C4 | 374 | 73.08% | 2.193 | 3.180 |
| C8 | 665 | 72.21% | 2.166 | 3.156 |
| C16 | 1,176 | 72.71% | 2.181 | 3.188 |
| C24 | 1,595 | 73.42% | 2.203 | 3.202 |
Both measured short-output grids reach their highest aggregate throughput at C24. C24 long-reasoning runs recorded timeouts, so C24 is not recommended as a validated long-output optimum. Acceptance is accepted / proposed draft tokens; it differs from accepted draft length and final completion length per verification. Dividing completion length by 4 does not give acceptance.
Choose and download
| Directory | Quantization | Complete package size |
|---|---|---|
fast/ |
W4A4 NVFP4, retained BF16 head/control components | 20.62 GB |
quality/ |
W4A4 NVFP4 + W8A8 FP8, retained BF16 head/control components | 25.80 GB |
Choose mixed precision when prioritizing the capability scores in this evaluation. Fast is smaller and has higher throughput in this C24 short-output sweep. Package size is not VRAM usage. Both are safetensors packages, not GGUF; vision and MTP remain BF16 rather than NVFP4. The decoder is official native MTP, not DFlash2.
Download all 15 files in the chosen directory: model-nvfp4-fast.safetensors (fast/) or model-nvfp4-mixed.safetensors (quality/), vision-mtp-bf16.safetensors, configs/index/tokenizer/processors, manifest.json, and SHA256SUMS. Do not download only the text shard or mix variant files. Each variant contains 64 text layers, 333 BF16 vision tensors, and 15 BF16 MTP tensors. The official MTP was not trained during this SFT/SimPO run.
Known boundaries
| C24 observation | Fast | Mixed precision |
|---|---|---|
| GPQA request timeouts | 12 | 11 |
| GPQA output-limit returns / empty final among returned requests | 14 / 14 | 13 / 13 |
| MMLU request errors / empty finals | 0 / 0 | 0 / 0 |
| LCB request errors | 0 | 0 |
| LCB output-limit returns / empty code | 25 / 26 | 25 / 25 |
These counters overlap and must not be added as disjoint failures. Scores follow the frozen scorer's final decisions; an empty visible final is a separate observation. We do not claim zero strict loops or losslessness in every mode. A fast-variant non-thinking code probe answered incorrectly in both bare and MTP modes; its XH counterpart passed. Mixed-precision MTP is validated; bare mode is not yet verified.
Training method
Official Qwen3.8-27B → capability-preserving SFT → terminal-behavior SimPO → per-tensor FP32 delta merge → BF16 → two NVFP4 exports.
- SFT: 1,905 samples, 1 epoch, 239 optimizer steps, effective batch 8; LoRA r16 / alpha32 / dropout 0.05; LR 5e-6 and 12 warmup steps.
- SimPO: 110 preference pairs, 73 unique prompts, 5 optimizer steps; beta 1.0, gamma 0.2, peak LR 5e-7; LoRA r16 / alpha32 / dropout 0; two-GPU FSDP full sharding.
- Training hardware: 2× RTX PRO 6000 Blackwell Server Edition.
SGLang runtime notes
Use the validated SGLang native-MTP path; vLLM has not been validated with this package. The frozen Spark launch records use the settings below; do not interchange the quantization backends:
| Setting | Fast | Mixed precision |
|---|---|---|
--quantization |
modelopt_fp4 |
modelopt_mixed |
--speculative-algorithm |
EAGLE |
EAGLE |
| steps / top-k / draft tokens | 3 / 1 / 4 | 3 / 1 / 4 |
| KV / context | BF16 / 65,536 | BF16 / 65,536 |
That frozen environment uses SGLang source revision 17313cf4b25d with runtime adaptations; this is not a clean-upstream-install compatibility guarantee. The formal scores and short sweeps above were measured on PRO 6000, not Spark. Do not reuse GGUF DFlash2 arguments or private container-image names.
Only the frozen test environment is confirmed to run; a portable clean-install environment still requires validation. This repository does not provide a launch script that depends on an unavailable private local container image. This page retains the C24 capability report.
Qwen3.8-27B EfficientThink — NVFP4 + BF16 MTP
科研免责声明： 本实验版本仅用于研究拒答倾向消融的技术可行性及行为影响，不构成全面的安全结论、对无限制使用的认可或专业建议。用户应依法、恰当地使用，并独立核验模型输出。
图中比较本模型的极速版与混合精度版，不是原版与后训练版对比。GPQA 思考统计仅含正常返回的 186 / 187 题，正确率仍保留全部 198 题；MMLU 和 LCB 思考统计分别含 500 / 100 题。本页能力成绩继续使用已冻结的 C24 结果。
三项正式成绩
| 版本 | GPQA 198 | MMLU 500 | LCB 100 |
|---|---|---|---|
| NVFP4 极速版 | 158/198（79.80%） | 430/500（86.00%） | 74/100（74.00%） |
| NVFP4 混合精度版 | 168/198（84.85%） | 449/500（89.80%） | 74/100（74.00%） |
实测环境为单张 RTX PRO 6000 96GB、SGLang、官方 BF16 MTP、C24、xhigh、32,768 输出预算、1,800 秒请求超时。GPQA 的 12 / 11 个超时均计入失败分母；未丢弃异常题。混合精度版比极速版多答对 10 道 GPQA 和 19 道 MMLU，LCB 总分相同。不同运行格式及解码器的结果不能直接当作等条件量化损失。
混合精度版相对极速版：得分与思考量
| 指标 | W4A4 极速版 | W4A4 + W8A8 混合精度版 | 变化 |
|---|---|---|---|
| GPQA 得分 | 158/198 (79.80%) | 168/198 (84.85%) | +10题 / +5.05pp |
| GPQA 平均思考量 | 9,123 | 8,400 | -7.9% |
| GPQA P50 / P90 | 4,928.5 / 24,893.0 | 4,278.0 / 24,402.6 | — |
| MMLU 得分 | 430/500 (86.00%) | 449/500 (89.80%) | +19题 / +3.80pp |
| MMLU 平均思考量 | 963 | 848 | -11.9% |
| MMLU P50 / P90 | 225.0 / 2,285.5 | 216.5 / 1,668.9 | — |
| LCB 得分 | 74/100 (74.00%) | 74/100 (74.00%) | 持平 |
| LCB 平均思考量 | 14,602 | 13,647 | -6.5% |
| LCB P50 / P90 | 9,626.5 / 32,769.0 | 7,441.5 / 32,769.0 | — |
同一轮 C24 协议下，混合精度版 GPQA 多答对 10 题、MMLU 多答对 19 题，LCB 得分持平；三项平均思考 token 均减少。这里比较的是量化版本，不是原版与后训练版。GPQA 思考统计仅含正常返回的 186 / 187 题，正确率分母始终为 198；MMLU/LCB 均值分母为 500/100，百分比变化从未四舍五入的均值计算。
MTP 并发速度与接受率
以下是既有256-token 定长短输出测试，每档单批，两版各 53 个请求。聚合吞吐不是单用户速度，也不是 32K 长推理持续吞吐。这里的 C16 是已完成的短输出并发测速，不代表 C16 全量能力评分。
极速版
| 并发 | 聚合 tok/s | MTP 接受率 | 接受草稿 / 验证轮 | 实际输出 / 验证轮 |
|---|---|---|---|---|
| C1 | 82 | 75.64% | 2.269 | 3.282 |
| C4 | 318 | 76.56% | 2.297 | 3.303 |
| C8 | 688 | 74.49% | 2.235 | 3.225 |
| C16 | 1,196 | 74.25% | 2.227 | 3.223 |
| C24 | 1,684 | 73.17% | 2.195 | 3.197 |
混合精度版
| 并发 | 聚合 tok/s | MTP 接受率 | 接受草稿 / 验证轮 | 实际输出 / 验证轮 |
|---|---|---|---|---|
| C1 | 102 | 72.92% | 2.188 | 3.200 |
| C4 | 374 | 73.08% | 2.193 | 3.180 |
| C8 | 665 | 72.21% | 2.166 | 3.156 |
| C16 | 1,176 | 72.71% | 2.181 | 3.188 |
| C24 | 1,595 | 73.42% | 2.203 | 3.202 |
两版已测短输出网格均在 C24 取得最高聚合吞吐，但 C24 长推理出现过超时，因此暂不推荐它作为长输出最佳并发。接受率按实际接受草稿数 / 提议草稿数计算；它与每轮接受长度、每轮最终输出不同，不能用输出长度除以 4 替代。
选择与下载
| 目录 | 量化方案 | 完整包大小 |
|---|---|---|
fast/ |
W4A4 NVFP4，保留 BF16 头部与控制部件 | 20.62 GB |
quality/ |
W4A4 NVFP4 + W8A8 FP8，保留 BF16 头部与控制部件 | 25.80 GB |
混合精度版更适合优先考虑本轮能力成绩的使用者；极速版文件较小，在本次 C24 短输出下吞吐更高。文件大小不是显存需求。两版都是 safetensors 包，不是 GGUF；视觉与 MTP 均为 BF16，而非 NVFP4。MTP 使用官方原生 MTP，不是 DFlash2。
下载所选目录的完整 15 个文件：model-nvfp4-fast.safetensors（fast/）或 model-nvfp4-mixed.safetensors（quality/）、vision-mtp-bf16.safetensors、配置/索引/tokenizer/processor、manifest.json 和 SHA256SUMS。不要只下载文字分片，也不要把两版文件混放。每版含 64 层文字主干、333 个 BF16 视觉张量及 15 个 BF16 MTP 张量；官方 MTP 未参与本次 SFT/SimPO 训练。
已知边界
| C24 观测 | 极速版 | 混合精度版 |
|---|---|---|
| GPQA 请求超时 | 12 | 11 |
| GPQA 达到输出上限 / 已返回请求空 final | 14 / 14 | 13 / 13 |
| MMLU 请求错误 / 空 final | 0 / 0 | 0 / 0 |
| LCB 请求错误 | 0 | 0 |
| LCB 达到输出上限 / 空代码 | 25 / 26 | 25 / 25 |
同一题可同时进入多个计数，不能相加为互斥失败数。得分采用冻结判分器最终结果，空可见 final 是单独的观测项。当前不宣称严格死循环为零或全模式无损。极速版不思考代码探针曾在 bare 和 MTP 模式均答错，XH 对应探针答对。混合精度版 MTP 已验证，bare 模式尚未验证。
训练方法
官方 Qwen3.8-27B → 能力保持 SFT → 终止行为 SimPO → 逐张量 FP32 增量合并 → BF16 → 两版 NVFP4 导出。
- SFT：1,905 条样本，1 epoch，239 个优化器步，有效 batch 8；LoRA r16 / alpha32 / dropout 0.05；LR 5e-6、12 步 warmup。
- SimPO：110 对偏好数据、73 个唯一 prompt，5 个优化器步；beta 1.0、gamma 0.2、峰值 LR 5e-7；LoRA r16 / alpha32 / dropout 0，双卡 FSDP full sharding。
- 训练硬件：2× RTX PRO 6000 Blackwell Server Edition。
SGLang 运行说明
推荐使用已验证的 SGLang 原生 MTP 路径；vLLM 未在本包验证。冻结的 Spark 启动记录使用以下参数，不能把两版的量化后端混用：
| 设置 | 极速版 | 混合精度版 |
|---|---|---|
--quantization |
modelopt_fp4 |
modelopt_mixed |
--speculative-algorithm |
EAGLE |
EAGLE |
| steps / top-k / draft tokens | 3 / 1 / 4 | 3 / 1 / 4 |
| KV / context | BF16 / 65,536 | BF16 / 65,536 |
该冻结环境使用 SGLang 源码版本 17313cf4b25d，并带运行环境适配；它不是干净上游安装的兼容性保证。上方正式得分和短扫则来自 PRO 6000，不能当作 Spark 的测速。不要照搬 GGUF DFlash2 参数或私有镜像名。
当前仅确认冻结测试环境可运行；便携全新安装环境仍需核验。仓库不提供无法公开获取的本机私有镜像启动脚本。本页继续保留 C24 能力报告。