library_name: transformers
tags:

• text-generation-inference
• llama-cpp
• mtp
• Speculative Decoding
• abliterated
• huihui
• qwen3.8
• multimodal
• vision-language-model
• image-text-to-text
• image-understanding
• visual-question-answering
• visual-reasoning
• document-understanding
• OCR
• image-captioning
• video-captioning
• video-understanding
  license: apache-2.0
  language:
• en
  base_model:
• prithivMLmods/Qwen3.8-27B-abliterated-GGUF
  pipeline_tag: image-text-to-text

Qwen3.8-27B-abliterated-MTP-GGUF

Qwen3.8-27B-abliterated-MTP-GGUF is a BF16 model checkpoint with Multi-Token Prediction (MTP) support, designed for efficient conversion into GGUF quantization formats for local deployment. The model removes refusal behavior through abliteration, a proof-of-concept activation-editing technique that modifies model weights without relying on TransformerLens. Built on the Qwen3.5 architectural foundation, Qwen3.8-27B is a 27-billion-parameter dense causal language model featuring a native vision encoder, a 64-layer hybrid architecture that interleaves Gated DeltaNet linear-attention blocks with periodic Gated Attention layers, and a native 262,144-token context window that can be extended to 1M tokens using YaRN. It supports native image and video understanding, flexible thinking control through the reasoning_effort parameter, and Multi-Token Prediction (MTP) for speculative decoding workflows.

[!NOTE]
MTP — Speculative Decoding: Yes

[!NOTE]
Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.

Model Files

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Quantized using llama.cpp by prithivMLmods