antonyMox/Qwen3.6-27B-abliterated-AutoRound-INT4-MTP

license: apache-2.0
base_model: Qwen/Qwen3.6-27B
base_model_relation: quantized
library_name: vllm
pipeline_tag: text-generation
language:

• en
• zh
• ru
  tags:
• qwen3.6
• int4
• autoround
• abliterated
• uncensored
• mtp
• speculative-decoding
• vllm
• long-context
• coding
• multimodal
• vision
  model-index:
• name: Qwen3.6-27B-abliterated-AutoRound-INT4-MTP
  results:
  • task:
    type: text-generation
    name: Code generation
    dataset:
    name: LiveCodeBench-hard (subset, reproducible harness)
    type: livecodebench-hard-subset
    metrics:
    • type: pass@1
      value: 70.0
      name: hard pass@1 (%)
      verified: false
  • task:
    type: text-generation
    name: Tool use
    dataset:
    name: Tool-use tasks (own harness)
    type: tool-use-own
    metrics:
    • type: pass@1
      value: 87.0
      name: tool-use pass@1 (%)
      verified: false

A gentle Heretic abliteration of the pristine Qwen/Qwen3.6-27B
→ AutoRound INT4 → the 15-tensor MTP head grafted back in BF16 so speculative decoding
works out of the box in vLLM. Removing refusals cost zero hard-coding points (70 = 70 vs the
clean base) and tool-use went up (87 vs 83). Pick a preset, run one script, done.

Highlights

Benchmarks — decensoring cost nothing, and where it sits

Same base Qwen/Qwen3.6-27B, one harness. Decensoring left hard unchanged (70 = 70 vs the
reference narrow quant) and tool-use went up (83 → 87):

[!NOTE]
Numbers are % of tasks passed (40 hard coding + 30 tool tasks; raw pass/total in
benchmark-results/). Conditions: temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 ·
seed 42 · non-thinking · vLLM 0.19 · RTX 4090 (48 GB) · MTP n=4. Solutions run in a restricted
sandbox (no network) — safe, identical scoring. The Lorbus row is our own measurement of their
public quant on the same tasks, not their claim. The 80B Coder-Next row is likewise our own
measurement of the UD-IQ4_XS GGUF (via LM Studio) on the same task set — a bigger, different model shown
for scale. Reproduce it, don't trust it.

Surgically gentle abliteration

Abliteration strips the refusal direction — done carelessly it also scrambles the model's brains.
Heretic co-minimizes refusals and KL divergence from
the original. Lower KL = closer to the untouched model = intelligence kept. We tuned for minimum drift:

~18× below Heretic's own showcase, 50–100× below manual. The model barely moved from the base,
so <think> reasoning and coding stayed intact. (KL is model/harness-dependent, so cross-model
figures aren't a strict race — the order-of-magnitude gap is the point.) We kept 11/100 refusals as
the price of rock-bottom drift — brains first.

Kept in BF16 (not quantized) — surgical precision

Everything heavy is 4-bit; only the fragile bits that carry reasoning, speed and vision stay full-precision.

Speed — MTP speculative decoding

Honest 768-token × 3-run measurement on a single RTX 4090 (48 GB), vLLM 0.19:

Use num_speculative_tokens: 4. Speculative decoding is lossless — n changes speed only,
never output. All presets use n=4.

Multimodal — sees images & video

The full vision-language model, not a text-only trim. The entire vision tower (333 tensors)
is kept in full BF16 — quantization never touched it — so image/video understanding is
identical to the pristine base. Serve with --limit-mm-per-prompt '{"image":4,"video":1}'.
Published benchmarks cover text/coding/tool-use; vision quality is inherited, not separately re-scored.

Quickstart — pick your GPU

vllm serve antonyMox/Qwen3.6-27B-abliterated-AutoRound-INT4-MTP \
  --quantization auto-round \
  --max-model-len 40960 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e4m3 \
  --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":4}' \
  --trust-remote-code

Long context is cheap here: Qwen3.6 is a hybrid (Mamba/GDN + attention), so KV costs only ~37 MB/1k
tokens at fp8. The model's native maximum is 256k (262,144) tokens — how much of it fits depends
on how much VRAM you give vLLM:

Context vs mem-util (48 GB card, measured)

Our default preset is a deliberate 0.60 / 200k — it keeps ~19 GB free so you can load a second
model on the same card. Want the full 256k? Just raise --gpu-memory-utilization to ~0.65–0.68.

⚠️ Pitfall — never set --max-model-len at exact KV capacity

[!WARNING]
On vLLM 0.19 hybrid models: if max-model-len equals KV capacity (startup log shows
Maximum concurrency ... : 1.00x), requests in the last ~3 % of the window pass validation
then hang forever — no response, no error, nothing logged, GPU idle. The scheduler can't
allocate the final KV block (draft tokens need headroom the validator ignores). Above the window →
instant HTTP 400 (fine); below the danger zone → fine. Only the top sliver dead-locks.

[!TIP]
Keep --max-model-len ≥ 2 attention blocks (2 × 1632) below capacity — our presets already do.
Check the log: Maximum concurrency for N tokens: 1.13x must be ≥ ~1.10x, not 1.00x.
Also: the log line GPU KV cache size: X tokens understates real capacity ~3× on hybrids
(it divides by KV-cache groups) — trust only the Maximum concurrency line.

Long-context recall (measured)

Needle test on a 150k-token prompt (48 GB preset), exact-quote recall by depth:

7/8 exact. The miss (*) at 50% returned the neighbouring line — mild "lost-in-the-middle",
not a hallucination. Prefix caching works: a repeat 150k query answered in 5.5 s vs 131 s cold.

🧬 The family

Clean quants of the same base — pick by hardware:

• wide · 26 GB · hard 78 🏆 — maximum reasoning, for 48 GB+ cards.
• abliterated · 19 GB · hard 70 (this one) — uncensored, fits 24 GB cards.
• 35B-A3B · 22 GB · hard 48 (coming soon) — fast MoE (~188 tok/s).

All ours: AutoRound INT4, MTP preserved, multimodal, published benchmarks. Measured against the
community reference Lorbus narrow INT4
(hard 70) — see the comparison table above.

Recipe

1. Base — pristine Qwen/Qwen3.6-27B BF16.
2. Abliteration — Heretic, gentle: KL 0.0089, refusals 11/100, <think> intact.
3. Quant — AutoRound W4A16, group 128, 200 iters, dense. BF16-kept: mtp.*, visual.*, Mamba control.
4. MTP — 15 mtp.* tensors grafted from the pristine BF16.
5. Sampler — ships temp 1.0 / top_p 0.95 / top_k 20 (generation_config.json).

Why HuggingFace shows "~6.7B params"

Not a small model — it's how the widget reads a packed INT4 quant. The 800 quantized tensors
pack 8× int4 into each int32, so HF counts 3.06 B int32 slots, not the ~24 B real int4 weights.
With the BF16 parts, the true model is Qwen3.6-27B, 19 GB. Every AutoRound/GPTQ INT4 looks like
this. Verified: language_model 4.55 B · other 1.27 B · vision 0.46 B · mtp 0.42 B.

⚠️ Usage & responsibility

[!CAUTION]
This is an uncensored research artifact — read before use.

• Reduced refusals by design — can produce content a safety-aligned model would decline.
• No safety guarantees — outputs are not filtered; review before use, especially in production.
• You are responsible — for lawful, ethical use in your jurisdiction; all consequences rest with you.
• No liability — provided "as is" as a community artifact, no warranty of any kind.
• Not for harm — for research, evaluation, and lawful applications only.

License

Apache-2.0, same as the base. Quantized & abliterated from
Qwen/Qwen3.6-27B. Community build; benchmark numbers
from our own harness — JSON included, methodology open.