license: apache-2.0
base_model:
- Kwaipilot/KAT-Coder-V2.5-Dev
tags: - kat-coder
- kat-coder-v2.5
- qwen3.5
- moe
- coding
- gguf
- llama.cpp
- mtp
- speculative-decoding
- abliteration
- uncensored
language: - en
- zh
KAT-Coder V2.5 Dev 35B-A3B MTP Abliterated
KAT-Coder V2.5 Dev 35B-A3B with a transplanted Multi-Token Prediction (MTP) head, based on the abliteration / "Philadelphia Class" variant of KAT-Coder V2.5 Dev.
This repository contains:
- The merged Hugging Face-format checkpoint
- A BF16 GGUF
- A Q4_K_M GGUF
- A transplanted single-layer MTP head for llama.cpp
draft-mtpspeculative decoding
Model lineage
Original model
Abliterated base
KridgeDookie/KAT-Coder-V2.5-Dev-35B-A3B-ABLITERATED-UNCENSORED-PHILADELPHIA-CLASS
MTP head
Myric/KAT-Coder-V2.5-Dev-MTP-head
The Philadelphia Class checkpoint is used as the target/base model, with the MTP tensors from Myric's KAT-Coder V2.5 Dev MTP head transplanted into it.
MTP modification
The source Philadelphia Class model has:
mtp_num_hidden_layers = 0
This merged checkpoint changes it to:
mtp_num_hidden_layers = 1
and adds the 19 MTP tensors from the MTP-head checkpoint.
The resulting Hugging Face weight layout is:
| File | Tensor count |
|---|---|
model.safetensors |
693 |
model-00014-of-mtp.safetensors |
19 |
| Total | 712 |
GGUF files
KAT-Philly-MTP-BF16.ggufKAT-Philly-MTP-Q4_K_M.gguf
The GGUF conversion exposes the transplanted MTP layer as the additional blk.40 / nextn tensors expected by llama.cpp.
llama.cpp usage
A recent llama.cpp build with Qwen3.5 MoE MTP support is required.
Example:
llama-server \
-m KAT-Philly-MTP-Q4_K_M.gguf \
--host 0.0.0.0 \
--port 8080 \
--parallel 1 \
--reasoning-preserve \
--fit-ctx 262144 \
--fit on \
--flash-attn on \
--cache-type-k q4_0 \
--cache-type-v q4_0 \
--cache-type-k-draft q4_0 \
--cache-type-v-draft q4_0 \
--spec-type draft-mtp \
--spec-draft-n-max 2
Validation
The model has been successfully tested with llama.cpp draft-mtp.
The converted model reports:
architecture: qwen35moe
model type: 35B.A3B
parameters: 35.51 B
training context: 262144
experts: 256
experts used/token: 8
nextn predict layers: 1
Observed first-position MTP acceptance was generally around 70-76%. Second-position acceptance with --spec-draft-n-max 2 was generally around 48-55%.
Performance note
Testing on:
GPU: NVIDIA GeForce RTX 3080 Ti, 12 GB
CPU: AMD Ryzen 7 3700X
RAM: 48 GB
showed that MTP worked correctly, but the additional MTP working set reduced the amount of the MoE model that could remain resident in VRAM.
Representative results:
| Configuration | Prompt processing | Sustained generation |
|---|---|---|
| Non-MTP KAT Q4_K_M | ~537 t/s | ~38.4 t/s |
| MTP Q4_K_M, n_max=2 | ~350-465 t/s | ~29-30 t/s |
| MTP Q4_K_M, n_max=1 | ~411 t/s | ~26.8 t/s |
Systems with substantially more VRAM may behave differently.
Credits
- Kwaipilot — KAT-Coder V2.5 Dev
- KridgeDookie — Abliterated / Uncensored Philadelphia Class variant
- Myric — KAT-Coder V2.5 Dev MTP head
- llama.cpp contributors — GGUF conversion, quantization, and MTP inference support