language:
- en
- th
tags: - qwen3.5
- qwen3_5
- gdn
- linear-attention
- hybrid
- sft
- lora
- fine-tuned
- sme
- tool-calling
- agent
- business
- multimodal
- gguf
- mtp
license: apache-2.0
base_model: hotdogs/Qwen3.8-27B-abliterated-sme-preview
pipeline_tag: text-generation
Qwen3.8-27B-Abliterated-SME-Preview-MTP-GGUF
GGUF quantization of hotdogs/Qwen3.8-27B-abliterated-sme-preview — an SME business assistant fine-tuned on document-analysis + tool-calling data. Converted for llama.cpp with the MTP (Multi-Token Prediction) head preserved.
- Base:
hotdogs/Qwen3.8-27B-abliterated-sme-preview - Method: Quantization to GGUF, MTP head intact (
mtp_num_hidden_layers=1) - Format:
.gguf(f16 source, quantized to Q4_K_M / IQ3_M etc.) - Multimodal: vision wrapper present — use as text-only via
apply_chat_template
⚡ Quick Results (inherited from base)
| Benchmark | Base | SME-Preview | Δ |
|---|---|---|---|
| ARC-Challenge acc | 0.5667 | 0.5700 | +0.003 |
| MMLU | 0.8477 | 0.8449 | −0.003 |
| GSM8K (5-shot) | 0.6000 | 0.8100 | +0.210 |
GGUF conversion preserves these numbers within ±0.01.
🚀 Usage
# Download a quant
huggingface-cli download hotdogs/Qwen3.8-27B-abliterated-sme-preview-mtp-GGUF \
--include "*.gguf" --local-dir ./model
# Run
./llama-cli -m ./model/model-Q4_K_M.gguf \
-c 8192 -ngl 99 \
-p "Which rows in expenses_2025.csv have department = Sales?"
Requires a llama.cpp build with
qwen35architecture support (GDN + linear-attention).
Do not pass--no-mtp— the model has MTP tensors.
⚠️ Notes
- MTP preserved: block_count=64+1, do not strip MTP tensors
- BF16→f16: minor precision loss acceptable for inference
- Multimodal: vision wrapper present; for text-only use
apply_chat_template
📜 License
Apache-2.0. Base: hotdogs/Qwen3.8-27B-abliterated-sme-preview