license: apache-2.0
base_model:
- llmfan46/Qwen3.6-27B-uncensored-heretic-v2
- Qwen/Qwen3.6-27B
pipeline_tag: image-text-to-text
tags: - heretic
- abliteration
- multimodal
- mtp
- speculative-decoding
- bf16
base_model_relation: merge
huginnfork/Qwen3.6-27B-uncensored-heretic-v2-mtp
bf16 build derived from the heretic-abliterated llmfan46/Qwen3.6-27B-uncensored-heretic-v2 of Qwen/Qwen3.6-27B, with the MTP head and vision tower preserved.
Provenance
- Base:
Qwen/Qwen3.6-27B(bf16) - Abliteration tool: heretic v1.2.0 (ARA) by Philipp Emanuel Weidmann (
p-e-w) - Abliterated source weights:
llmfan46/Qwen3.6-27B-uncensored-heretic-v2— the heretic-derived bf16 abliteration ofQwen/Qwen3.6-27B - MTP head: re-grafted from
Qwen/Qwen3.6-27B(15 tensors, ~810 MB bf16) so SGLang/vLLM speculative decoding (--speculative-algo NEXTN) works - Vision tower:
model.visual.*preserved in bf16 (333 tensors)
KL divergence measurements
KLD computed with eval_kld.py — per-token KLD averaged over 8 samples from neuralmagic/calibration (LLM split), max_seq=1024. Max sample KLD is the highest single-sample mean (catches outliers that the overall mean hides).
| Comparison | Mean KLD (nats) | Max sample KLD | Samples | max_seq |
|---|---|---|---|---|
| vs Qwen3.6-27B base | 0.0425 | 0.1104 | 8 | 1024 |
Note: this pipeline always uploads the resulting checkpoint. Consult the KL
divergence numbers above to judge whether the result is acceptable for your
use case.
Perplexity (wikitext-2-raw)
Wikitext-2-raw test split, non-overlapping chunks of 2048 tokens, computed with eval_ppl.py. Same tokenizer for every row so the numbers compare apples-to-apples.
| Model | Perplexity | Tokens scored | Dataset | seq |
|---|---|---|---|---|
| Qwen3.6-27B base (bf16) | 7.3057 | 296907 | wikitext/wikitext-2-raw-v1/test |
2048 |
| this checkpoint | 7.4619 | 296907 | wikitext/wikitext-2-raw-v1/test |
2048 |
Inference
transformers (text + vision; MTP not exercised)
from transformers import AutoModelForImageTextToText, AutoProcessor
import torch
repo = "huginnfork/Qwen3.6-27B-uncensored-heretic-v2-mtp"
proc = AutoProcessor.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForImageTextToText.from_pretrained(
repo, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)
vLLM (bf16 + MTP speculative decoding)
vllm serve huginnfork/Qwen3.6-27B-uncensored-heretic-v2-mtp \
--trust-remote-code \
--gpu-memory-utilization 0.85 \
--max-model-len 8192 \
--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":1}'