base_model: bottlecapai/ThinkingCap-Qwen3.6-27B
base_model_relation: quantized
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
language:

• en
• zh
  tags:
• nvfp4
• fp4
• w4a4
• gptq
• quantized
• compressed-tensors
• llm-compressor
• vllm
• qwen3_5
• vision-language
• thinking
• mtp
• speculative-decoding
• uncensored
• abliterated
• heretic

TL;DR: ThinkingCap-Qwen3.6-27B abliterated, quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. 18 GB, wikitext-2 PPL 7.25, 256K thinking VL, refusals removed (2-round Heretic). The MTP (multi-token prediction) head is kept at bf16 so speculative decoding runs at full precision.

ThinkingCap-Qwen3.6-27B abliterated NVFP4

bottlecapai/ThinkingCap-Qwen3.6-27B,
abliterated (refusal direction removed) with Heretic in
two iterative rounds, then quantized to NVFP4 (W4A4) in the compressed-tensors
nvfp4-pack-quantized format with llm-compressor
(GPTQ + MSE, shared fused-layer scales), with the MTP head left at bf16.

Near-lossless, decensored, and speculative-decoding-ready. Two Heretic rounds cut refusals from
99/100 to 8/100 while keeping a KL divergence of 0.0408 to the original model (well under the
0.5 line that signals capability damage). NVFP4 compresses to ~18 GB at wikitext-2 perplexity 7.25.
The multi-token-prediction head that drives self-speculative decoding is excluded from
quantization and kept at bf16, so the speedup it provides is not degraded by 4-bit draft
predictions.

• Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs
  run it weight-only.
• Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).

Uncensored / abliterated model. It follows instructions without refusal guardrails. The
abliteration only removes refusals; all other behaviour comes from the base model. You
are responsible for how you use it.

Why the MTP head is kept at bf16

ThinkingCap ships a multi-token prediction (MTP) head (mtp_num_hidden_layers: 1), a small
extra module that predicts the next token(s) so the model can self-speculate: it drafts ahead
and verifies in one pass, cutting decode latency without a separate draft model.

That head is on the critical path for acceptance rate. If it were quantized to 4-bit, its draft
predictions would drift from what the full-precision main model would have produced, the verifier
would reject more drafts, and the speculative speedup would shrink, you would pay the MTP compute
and get less of the latency back. So the quant recipe ignores re:.*mtp.* and leaves the head
at bf16 (it is tiny relative to the 27B backbone, so the size cost is negligible). The 4-bit main
model does the heavy lifting; the full-precision MTP head keeps acceptance high.

Fidelity

Near-lossless versus the bf16 source, 18 GB vs ~55 GB bf16 (~33%), at wikitext-2 perplexity 7.25 and KL divergence 0.0408 to the original. GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.

Quickstart

NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag
needed. --reasoning-parser qwen3 splits the <think> block into reasoning_content.

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --reasoning-parser qwen3

• Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
  --max-model-len 131072 is a safe default; raise it if memory allows.
• Add --language-model-only to skip the vision tower and free KV cache for text use.
• The parser flag is not auto-detected; pass it explicitly.

Speculative decoding (MTP)

The bf16 MTP head enables vLLM's built-in self-speculative decoding. Enable it with a speculative
config that uses the model's own next-token-prediction module (no separate draft model):

vllm serve maci0/ThinkingCap-Qwen3.6-27B-abliterated-NVFP4 \
  --served-model-name thinkingcap-27b-abliterated-nvfp4 \
  --max-model-len 131072 --kv-cache-dtype fp8 --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

num_speculative_tokens matches the head's depth (mtp_num_hidden_layers: 1). Measure acceptance
rate on your traffic; because the head is bf16, acceptance is not penalized by quantization.

Python (OpenAI client)

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="x")
r = client.chat.completions.create(
    model="thinkingcap-27b-abliterated-nvfp4",
    messages=[{"role": "user", "content": "Explain why a heap is better than a sorted array for a priority queue."}],
)
print(r.choices[0].message.content)

About the base model

A 27B Qwen3.5-family vision-language model (ThinkingCap-Qwen3.6-27B) with thinking-mode reasoning,
a 256K context window, and an MTP head for speculative decoding.

• 64 decoder layers: hybrid gated delta-net linear attention plus full attention, dense MLP,
  plus a vision tower for image and video input.
• MTP head (mtp_num_hidden_layers: 1) for multi-token / speculative decoding.
• 256K context (max_position_embeddings 262144).
• Thinking mode by default, with an instruct toggle.

Abliteration

Heretic removes the refusal direction with a TPE-optimized
search over per-component ablation strength, jointly minimizing refusal rate and KL divergence
from the original model, then merges the best trial. This model was abliterated in two iterative
rounds: round 1 removed the dominant refusal direction, then Heretic was re-run on the round-1
model to remove the residual refusal direction that surfaced once the first was gone. Because
ThinkingCap is a thinking model, evaluation ran in non-thinking mode so each judged response is a
real answer rather than an unfinished <think> block; the shipped model restores the original
thinking chat template.

• Datasets: mlabonne/harmless_alpaca (good) vs mlabonne/harmful_behaviors (bad).
• Final KL divergence 0.0408 (capability preserved, well under the 0.5 damage line).

Quantization

The recipe ignores re:.*mtp.*, re:.*visual.*, and lm_head. Keeping the MTP head at full
precision is the point: it preserves speculative-decoding acceptance (see above). GPTQ is a
quantization-time cost only; inference speed and format are identical to plain round-to-nearest
NVFP4, but it chooses better 4-bit values.

Calibration: 512 domain-matched samples (long reasoning + general chat + code),
max_seq_len=2048, text-only path through the VL model.

Recommended sampling

Thinking mode is the default.

• Thinking, precise: temperature=0.6, top_p=0.95, top_k=20
• Thinking, general: temperature=1.0, top_p=0.95, top_k=20
• Instruct / non-thinking: temperature=0.7, top_p=0.80, top_k=20
• To run non-thinking, set {%- set enable_thinking = false %} in the chat template, or
  pass extra_body={"chat_template_kwargs": {"enable_thinking": false}}.

Related

• Base model: bottlecapai/ThinkingCap-Qwen3.6-27B
• Space: Rogue Quants
• Collection: NVFP4 Quants
• Sibling NVFP4 quants:
  • Qwopus3.6-27B-Coder abliterated
  • Qwopus3.6-27B-v2 abliterated
  • Ornith-1.0-35B MoE abliterated
  • Qwen3.6-40B Deckard
  • Huihui-Qwythos-9B Claude-Mythos
  • Ornith-1.0-9B abliterated

Notes

• Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
• The MTP head is bf16; enable speculative decoding via --speculative-config to use it.
• --reasoning-parser is not auto-detected; pass it explicitly.
• Thinking mode is on by default; toggle it via the chat template or chat_template_kwargs.
• No refusal guardrails; you are responsible for how you use it.

License

Apache-2.0, following the base model. Intended use and all responsibility for use follow
the base model.

Credits

• Base model: bottlecapai
• Abliteration: Heretic by Philipp Emanuel Weidmann
• Quantization tooling: llm-compressor / compressed-tensors