license: apache-2.0
tags:
- transformers
- safetensors
- qwen3_5
- qwen3
- thinkingcap
- efficient-thinking
- reasoning
- token-efficient
- lora
- sft
- abliterated
- multi-token-prediction
- text-generation
base_model: - hotdogs/Qwen3.8-27B-abliterated
datasets: - hotdogs/thinkingcap-sft-qwen38-27b
pretty_name: Qwen3.8-27B ThinkingCap Abliterated Preview
Qwen3.8-27B ThinkingCap Abliterated Preview (Merged)
SFT-merged LoRA (checkpoint-42, epoch 1) from the
ThinkingCap efficient-reasoning dataset
(hotdogs/thinkingcap-sft-qwen38-27b).
- Base:
hotdogs/Qwen3.8-27B-abliterated(Qwen3_5, BF16, 55.6 GB) - LoRA: r=32, alpha=64, target = all linear + GDN in_proj layers
- Training: 339 SFT rows, 1 epoch, max_seq 4096, 7x RTX 3090, BF16
- Eval loss (epoch 1): 0.2143 (best of 3 tracked epochs)
- MTP: 15 tensors preserved in blk.64 (restored post-merge)
- Dataset:
hotdogs/thinkingcap-sft-qwen38-27b(369 SFT + 244 DPO pairs,
on-policy from live Qwen3.8-27B-Ablit, oracle-verified 84.2%) - GGUF versions:
hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview-mtp-GGUF
Files
19 files, 1199 tensor keys, 55.56 GB total.
Usage (transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview",
torch_dtype="auto", device_map="auto", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("hotdogs/Qwen3.8-27B-thinkingcap-abliterated-preview")
Smoke test (F16 GGUF)
- 27x43 = 1161 (think 163 chars)
- bat-and-ball = 0.05 (think 161 chars)
- snail wall = day 5 (think 359 chars, base was 4000+)