license: apache-2.0
language:

• en
• zh
  tags:
• qwen3.8
• gguf
• efficient-thinking
• dflash2
• speculative-decoding
• llama.cpp

Qwen3.8-27B EfficientThink GGUF

No strict loops were observed during evaluation.

BF16 / FP8 main repository: nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2

Measured quantization-tier overview

Capability scores are shown only after a formal full-suite freeze, with every non-passing sample retained in the denominator. Q8_0, Q6_K, Q5-LynnStyle, Q4-LynnStyle, and Q3-LynnStyle are published and have passed four-repository remote path, byte-size, and LFS-SHA256 readback.

Per-tier DFlash2 concurrency performance

Common measured protocol: one GPU for the main model, DFlash2, --parallel 8 --ctx-size 262144 (8 slots × 32K), C1/C2/C4/C8, fixed 256-token pressure outputs, and a 60-second request ceiling. DFlash2 used n_max=7 and n_min=0. Throughput is aggregate completion tok/s.

Q8_0

• Concurrency guidance: C4 practical balance; C8 maximum throughput.
• 15 fixed-length requests per tier; zero timeout, request error, empty output, or punctuation collapse.

Q6_K

• Concurrency guidance: C4 lower-latency balance; C8 maximum throughput.
• Zero timeout, request error, empty output, or punctuation collapse across the sweep.

Q5-LynnStyle

• Concurrency guidance: C4 practical balance; C8 maximum throughput.
• Zero timeout, request error, empty output, or punctuation collapse across the sweep.

Q4-LynnStyle

• Concurrency guidance: C4 practical balance; C8 maximum throughput.
• Zero timeout, request error, empty output, or punctuation collapse across the sweep.

Q3-LynnStyle

• Concurrency guidance: C4 is the practical balance; C8 is the maximum measured throughput.
• The full sweep had zero timeout, request error, empty output, or punctuation collapse.

Training method

Qwen/Qwen3.8-27B → capability-preserving SFT → terminal-behavior SimPO → per-tensor FP32 delta merge → BF16 → GGUF quantization.

• SFT: 1,905 samples, 1 epoch, 239 optimizer steps, effective batch 8; LoRA r=16, alpha=32, dropout=0.05; LR 5e-6, 12 warmup steps, seed 20260901.
• SimPO: 110 preference pairs over 73 unique prompts, 5 optimizer steps; beta=1.0, gamma=0.2, peak LR 5e-7; LoRA r=16, alpha=32, dropout=0, seed 20260903, world size 2, FSDP full sharding.
• Training hardware: 2x NVIDIA RTX PRO 6000 Blackwell Server Edition.

This repository publishes text-main GGUF files with independently verified vision and DFlash2 sidecars. Q8_0 and Q6_K use the Q8 vision projector; Q5-LynnStyle, Q4-LynnStyle, and Q3-LynnStyle use the Q6 vision projector. All five tiers share the same Q8-main-compatible BF16 DFlash2 sidecar; the DFlash2 file itself is not Q8-quantized.

Published GGUF files

Q8_0

Formal full-suite evaluation: GPQA 164/198 (82.83%) · MMLU 447/500 (89.40%) · LCB 74/100 (74.00%).

Concurrency guidance: C4 is the practical balance (246 tok/s, 60.98% acceptance); C8 is the maximum measured throughput (256 tok/s, 52.01%).

llama-server \
  --model Q8_0/Qwen3.8-27B-EfficientThink-SimPO-Q8_0.gguf \
  --mmproj Q8_0/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --model-draft Q8_0/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q5-LynnStyle

Formal full-suite evaluation: GPQA 164/198 (82.83%) · MMLU 438/500 (87.60%) · LCB 75/100 (75.00%). LCB: Easy 23/23, Medium 29/31, Hard 23/46.

Concurrency guidance: C4 is the practical balance (238 tok/s, 48.22% acceptance); C8 is the maximum measured throughput (276 tok/s, 48.38%).

llama-server \
  --model Q5-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q5-LynnStyle.gguf \
  --mmproj Q5-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q5-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q4-LynnStyle

Formal full-suite evaluation: GPQA 166/198 (83.84%) · MMLU 443/500 (88.60%) · LCB 74/100 (74.00%). LCB: Easy 23/23, Medium 30/31, Hard 21/46; 100 unique tasks, errors=0, clean gate PASS.

Concurrency guidance: C4 is the practical balance (257 tok/s, 47.69% acceptance); C8 is the maximum measured throughput (264 tok/s, 42.88%). The full sweep had zero timeout, request error, empty output, or punctuation collapse.

llama-server \
  --model Q4-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q4-LynnStyle.gguf \
  --mmproj Q4-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q4-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q3-LynnStyle

Formal full-suite evaluation: GPQA 172/198 (86.87%) · MMLU 435/500 (87.00%) · LCB 78/100 (78.00%). GPQA used 198 unique items with errors=0 and strict loop=0; MMLU had errors=0 and parse fail=0; LCB: Easy 23/23, Medium 28/31, Hard 27/46, with 100 unique tasks and errors=0. Every non-passing sample remains in the denominator.

Concurrency guidance: C4 is the practical balance (269 tok/s, 58.00% acceptance); C8 is the maximum measured throughput (286 tok/s, 52.41%). The full sweep had zero timeout, request error, empty output, or punctuation collapse.

llama-server \
  --model Q3-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q3-LynnStyle.gguf \
  --mmproj Q3-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q3-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

The Q8_0, Q5-LynnStyle, Q4-LynnStyle, and Q3-LynnStyle main GGUF files are GGUF v3, qwen35, 64-layer, 851-tensor text trunks with no embedded vision or MTP tensors; all passed remote byte/SHA256 readback and joint vision + DFlash2 smokes. Q5/Q4/Q3 were tested with the Q6 vision projector, and the four tiers share the BF16 DFlash2 draft.

LynnStyle mixed-precision quantization

Q5/Q4/Q3 LynnStyle tiers are quantized directly from the final SimPO F16 text trunk with the frozen 512-chunk imatrix (496 entries), without requantization. Protection combines T10 REAP, first/last-layer and SSM guards, plus per-tensor full-attention, SSM, and FFN audits; higher precision wins when rules overlap.

• Q5-LynnStyle (published): Q5 base; Q8 for L0–3, L60–63, and critical REAP layers 50/52/54/56/58; Q6 for high-sensitivity layers 21/42/44/46/48/49/53/57/59, full-attention q/k/v/o, and selected middle ssm_out. Of 498 converted tensors: Q8_0 104, Q6_K 186, Q5_K 208.
• Q4-LynnStyle (published): Q4 base; Q8 for L0–3 and L60–63; Q6 for critical shoulder layers 50/52/54/56–59; Q5 for high layers 21/42/44/46/48/49/53; Q6 for full-attention q/k/v/o and selected ssm_out. Distribution: Q8 62, Q6 156, Q5 172, Q4 108.
• Q3-LynnStyle (published): Q3 base; Q6 for L0–3 and L60–63; Q5 for L4–6 and 50/52/54/56–59; Q4 for high layers 21/42/44/46/48/49/53; Q5 for full-attention and middle ssm_out. Distribution: Q8 2, Q6 62, Q5 157, Q4 199, Q3 78.

Hub sidebars may expose only nominal GGUF general.file_type, which cannot fully describe custom per-tensor mixed precision. Treat each tier's manifest.json, SHA256SUMS, and actual GGUF parse as authoritative.

Q6_K

Formal full-suite evaluation: GPQA 171/198 (86.36%) · MMLU 440/500 (88.00%) · LCB 78/100 (78.00%).

LCB difficulty split: Easy 23/23, Medium 30/31, Hard 25/46.

Request errors, timeouts, empty outputs, and punctuation collapse were zero at all four tested concurrency levels. This serving sweep characterizes throughput and speculative acceptance; it does not replace formal quality evaluation.

Files

Recommended launch

llama-server \
  --model Q6_K/Qwen3.8-27B-EfficientThink-SimPO-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --model-draft Q6_K/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --split-mode layer --tensor-split 1,1 \
  --parallel 8 --ctx-size 32768 --flash-attn on

中文说明

测评中未发现严格死循环。

BF16 / FP8 主仓：nerkyor/Qwen3.8-27B-EfficientThink-Uncensored-K3-Opus5-Grok4.6-GPT5.6Sol-SFT-SimPO-DFlash2

量化档位实测总览

能力分数只列正式全量冻结结果，所有未通过样本均保留在分母。Q8_0、Q6_K、Q5-LynnStyle、Q4-LynnStyle 与 Q3-LynnStyle 均已发布，并完成四仓远端路径、字节和 LFS SHA256 回读。

分档 DFlash2 并发性能

统一实测口径：单卡主模型、DFlash2、--parallel 8 --ctx-size 262144（8 slots × 32K），C1/C2/C4/C8，256-token 定长压力输出，每请求最多 60 秒；DFlash2 使用 n_max=7、n_min=0。吞吐均为聚合 completion tok/s。

Q8_0

• **并发建议：**C4 实用平衡；C8 最大吞吐。
• 每档 15 个定长请求；超时、请求错误、空输出、标点坍塌均为 0。

Q6_K

• **并发建议：**C4 低延迟平衡；C8 最大吞吐。
• 整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

Q5-LynnStyle

• **并发建议：**C4 实用平衡；C8 最大吞吐。
• 整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

Q4-LynnStyle

• **并发建议：**C4 实用平衡；C8 最大吞吐。
• 整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

Q3-LynnStyle

• **并发建议：**C4 为实用平衡档；C8 为最大实测吞吐。
• 整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

训练方法

Qwen/Qwen3.8-27B → 能力保持 SFT → 终止行为 SimPO → 逐 tensor FP32 delta 合并 → BF16 → GGUF 量化。

• **SFT：**1,905 条样本，1 epoch，239 optimizer steps，effective batch 8；LoRA r=16、alpha=32、dropout=0.05；LR 5e-6，warmup 12 steps，seed 20260901。
• **SimPO：**110 组偏好对、73 个唯一 prompt，5 optimizer steps；beta=1.0、gamma=0.2、peak LR 5e-7；LoRA r=16、alpha=32、dropout=0，seed 20260903，world size 2，FSDP full sharding。
• **训练硬件：**双 NVIDIA RTX PRO 6000 Blackwell Server Edition。

本仓发布文本主 GGUF、视觉投影与经过独立验证的 DFlash2 draft。Q8_0 与 Q6_K 使用 Q8 视觉塔；Q5-LynnStyle、Q4-LynnStyle 与 Q3-LynnStyle 使用 Q6 视觉塔。五档共用同一份 Q8-main-compatible BF16 DFlash2 sidecar；DFlash2 文件本身不是 Q8 量化模型。

已发布 GGUF 文件

Q8_0

正式全量评测：GPQA 164/198（82.83%）· MMLU 447/500（89.40%）· LCB 74/100（74.00%）。

**并发建议：**C4 为实用平衡档（246 tok/s，60.98% 接受率）；C8 为最大实测吞吐（256 tok/s，52.01%）。

llama-server \
  --model Q8_0/Qwen3.8-27B-EfficientThink-SimPO-Q8_0.gguf \
  --mmproj Q8_0/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --model-draft Q8_0/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q5-LynnStyle

正式全量评测：GPQA 164/198（82.83%）· MMLU 438/500（87.60%）· LCB 75/100（75.00%）。LCB：Easy 23/23、Medium 29/31、Hard 23/46。

**并发建议：**C4 为实用平衡档（238 tok/s，48.22% 接受率）；C8 为最大实测吞吐（276 tok/s，48.38%）。

llama-server \
  --model Q5-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q5-LynnStyle.gguf \
  --mmproj Q5-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q5-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q4-LynnStyle

正式全量评测：GPQA 166/198（83.84%）· MMLU 443/500（88.60%）· LCB 74/100（74.00%）。LCB：Easy 23/23、Medium 30/31、Hard 21/46；100 个唯一题目、errors=0、clean gate PASS。

**并发建议：**C4 为实用平衡档（257 tok/s，47.69% 接受率）；C8 为最大实测吞吐（264 tok/s，42.88%）。整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

llama-server \
  --model Q4-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q4-LynnStyle.gguf \
  --mmproj Q4-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q4-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q3-LynnStyle

正式全量评测：GPQA 172/198（86.87%）· MMLU 435/500（87.00%）· LCB 78/100（78.00%）。GPQA 为 198 个唯一题目、errors=0、strict loop=0；MMLU errors=0、parse fail=0；LCB：Easy 23/23、Medium 28/31、Hard 27/46，100 个唯一题目、errors=0。所有未通过样本均保留在分母。

**并发建议：**C4 为实用平衡档（269 tok/s，58.00% 接受率）；C8 为最大实测吞吐（286 tok/s，52.41%）。整轮扫参中，超时、请求错误、空输出、标点坍塌均为 0。

llama-server \
  --model Q3-LynnStyle/Qwen3.8-27B-EfficientThink-SimPO-Q3-LynnStyle.gguf \
  --mmproj Q3-LynnStyle/mmproj-Qwen3.8-27B-Q6_K.gguf \
  --model-draft Q3-LynnStyle/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --parallel 8 --ctx-size 32768 --flash-attn on

Q8_0、Q5-LynnStyle、Q4-LynnStyle 与 Q3-LynnStyle 的主 GGUF 均为 GGUF v3、qwen35、64 层、851 tensors，主文件不内嵌视觉或 MTP tensor；均已通过远端字节/SHA256 回读与视觉 + DFlash2 联合 smoke。Q5/Q4/Q3 实测使用 Q6 视觉塔，四档共用 BF16 DFlash2 draft。

LynnStyle 混合精度量化

Q5/Q4/Q3 LynnStyle 均从最终 SimPO F16 文本主干直接量化，使用冻结的 512-chunk imatrix（496 entries），不做二次 requantize。保护策略综合 T10 REAP、首尾层/SSM 保护与 full-attention、SSM、FFN 的逐 tensor 审计；规则冲突时采用更高精度。

• **Q5-LynnStyle（已发布）：**基础 Q5；L0–3、L60–63 与关键 REAP 层 50/52/54/56/58 使用 Q8；高敏感层 21/42/44/46/48/49/53/57/59 使用 Q6；full-attention q/k/v/o 与选定中层 ssm_out 使用 Q6。498 个转换 tensor：Q8_0 104、Q6_K 186、Q5_K 208。
• **Q4-LynnStyle（已发布）：**基础 Q4；L0–3、L60–63 使用 Q8；关键肩部层 50/52/54/56–59 使用 Q6；高敏感层 21/42/44/46/48/49/53 使用 Q5；full-attention q/k/v/o 与选定 ssm_out 使用 Q6。分布：Q8 62、Q6 156、Q5 172、Q4 108。
• **Q3-LynnStyle（已发布）：**基础 Q3；L0–3、L60–63 使用 Q6；L4–6、50/52/54/56–59 使用 Q5；高敏感层 21/42/44/46/48/49/53 使用 Q4；full-attention 与中层 ssm_out 使用 Q5。分布：Q8 2、Q6 62、Q5 157、Q4 199、Q3 78。

Hub 侧边栏可能仅显示 GGUF 名义 general.file_type，无法完整表达自定义逐 tensor 混合精度；以各目录 manifest.json、SHA256SUMS 与实际 GGUF parse 为准。

Q6_K

正式全量评测：GPQA 171/198（86.36%）· MMLU 440/500（88.00%）· LCB 78/100（78.00%）。

LCB 难度分布：Easy 23/23、Medium 30/31、Hard 25/46。

四档固定输出测试的请求错误、超时、空输出和标点坍塌均为 0。吞吐测试不代替正式质量评测。

文件

推荐启动

llama-server \
  --model Q6_K/Qwen3.8-27B-EfficientThink-SimPO-Q6_K.gguf \
  --mmproj Q6_K/mmproj-Qwen3.8-27B-Q8_0.gguf \
  --model-draft Q6_K/dflash2-qwen38-27b-bf16.gguf \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --gpu-layers all --gpu-layers-draft all \
  --split-mode layer --tensor-split 1,1 \
  --parallel 8 --ctx-size 32768 --flash-attn on