license: apache-2.0
tags:

• transformers
• gguf
• qwen3
• thinkingcap
• efficient-thinking
• reasoning
• token-efficient
• lora
• sft
• abliterated
• multi-token-prediction
• mtp
• llama.cpp
• text-generation
  base_model:
• hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview
  model:
• hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview-mtp-GGUF
  datasets:
• hotdogs/thinkingcap-sft-qwen38-27b
  pretty_name: Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

Qwen3.8-27B ThinkingCap Abliterated (MTP GGUF)

llama.cpp GGUF from the ThinkingCap SFT-merged model
(hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview).

Trained from the on-policy oracle-verified dataset.

All 3 quantizations preserve the MTP layer (blk.64, 15 tensors
including 4 nextn.* projection tensors). 866 tensors total, 27.3B params.

Files

MTP verification

import gguf
g = gguf.GGUFReader("Qwen3.8-27B-thinkingcap-abliterated-preview-mtp-f16.gguf")
mtp = [t.name for t in g.tensors if 'blk.64' in t.name]
print(len(mtp))  # 15

Serve (llama.cpp)

CUDA_VISIBLE_DEVICES=0,1,2,3,4 llama-server \
  -m Qwen3.8-27B-thinkingcap-abliterated-preview-mtp-Q6_K.gguf \
  --n-gpu-layers 999 --ctx-size 8192 --parallel 2 \
  --batch-size 4096 --flash-attn on \
  --temp 1 --top-k 20 --top-p 0.95 --min-p 0.0 --jinja

📊 Benchmarks

Tested with inspect_ai 0.3.260 on local llama.cpp, reasoning_effort=low,
temperature=1, top_k=20, top_p=0.95. All runs are single-epoch (no multi-epoch averaging).

¹ Official vendor-reported scores from the Qwen3.8-27B model card, reasoning_effort=xhigh.
² Standard Google IFEval via inspect_evals/ifeval (541 prompts, prompt-level strict accuracy).
⁴ inspect_evals/swe_bench_verified_mini: 50 issues (25 django + 25 sphinx), message_limit=100 steps,
max_tokens=16384/turn, tool_timeout=210s, no internet, Docker sandbox.
⁶ External/independent evaluations (not the official model card). SWE-bench Verified (500) is a different, harder split than our 50-task mini. Treat as directional, not exact.
⁷ External comparison via serenitiesai.com (F16 reference values). Treat as directional.
⁸ From SmartScope Gemini 3.1 Pro Benchmark Analysis (2026).
GPT-5.5 and Kimi K3 SWE scores from MorphLLM SWE-bench Pro Leaderboard.

Reading it: Q6_K scores within 2.1 pp of the full BF16 Qwen3.8-27B on GPQA
(87.1 vs 89.2) — with a quantized, abliterated, LoRA-merged GGUF running on
consumer GPUs. On IFEval, Q6_K (88.2%) and Q4_K_M (88.4%) are near-identical and sit
within ~3 pp of the F16 reference (~91.5% for Qwen3.8-27B, 95.0% for Opus 4.6). On
SWE-bench, our 50-task mini run is retesting (previous 60% run had a harness issue);
external runs of the full 500-task SWE-bench Verified put Qwen3.8-27B at ~82% (footnote ⁶).

⚠️ Hardware limitation: Two-box setup — Box A: 1× RTX 3090 24GB + 1× RTX 3060 12GB (36 GB),
Box B: 1× RTX 4060 Ti 16GB + 1× RTX 3060 12GB (28 GB). Total 64 GB VRAM across
both boxes. F16 (55 GB) fits on Box A with tight KV-cache; Q6_K (21 GB) and
Q4_K_M (16 GB) are the practical daily-drivers. F16 / Q8_K benchmark runs are
pending — this table will be updated when they land.

⚠️ Token & step limits: SWE-bench is the most constrained benchmark:
message_limit=100 (agent turns) × max_tokens=16384/turn ≈ 1.6M output-token
ceiling. The model is typically step-limited, not token-limited (agent runs
out of turns before exhausting the token budget). GPQA is single-shot with
max_tokens=65536, so the token ceiling is rarely hit (avg ~4K tokens/question).

Smoke test

Base model (pre-SFT) used 4000+ think-chars on the snail problem.
ThinkingCap SFT brings it to 359 (-91%).