base_model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP
base_model_relation: quantized
license: apache-2.0
library_name: transformers
pipeline_tag: image-text-to-text
language:
- en
tags: - nvfp4
- fp4
- w4a4
- gptq
- quantized
- compressed-tensors
- llm-compressor
- vllm
- qwen3_5
- vision-language
- thinking
- mtp
- speculative-decoding
- uncensored
- heretic
| Params | 27B |
|---|---|
| Active | 27B (dense) |
| Size | 20 GB |
| Perplexity | n/a |
| Refusals | n/a |
| Context | 256K |
| MTP head | bf16 |
TL;DR: DavidAU's Qwen3.6-27B Fable-Fusion-711 (Uncensored/Heretic, with an MTP head), quantized to NVFP4 (W4A4) for vLLM on NVIDIA Blackwell. ~20 GB, 256K thinking VL, uncensored (from the base). The MTP head and vision tower are kept at bf16, 4-bit for the 27B backbone, full precision for speculative decoding and image input.
Qwen3.6-27B Fable-Fusion-711 NVFP4
DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP,
quantized to NVFP4 (W4A4) in the compressed-tensors nvfp4-pack-quantized format with
llm-compressor (GPTQ + MSE, shared fused-layer
scales). This is a quantization only, the model is already uncensored/heretic upstream; no
further abliteration was applied here.
- Built for vLLM on NVIDIA Blackwell (4-bit weight + 4-bit activation). Pre-Blackwell GPUs
run it weight-only. - The MTP head (
mtp.*) and vision tower (model.visual.*) are kept at bf16, so speculative
decoding and image input are unaffected by 4-bit. - Loading and generation verified in vLLM on an NVIDIA GB10 (Blackwell, sm_121).
Uncensored model (inherited from the base). It follows instructions without refusal guardrails.
You are responsible for how you use it.
Why the MTP head is kept at bf16
The base ships a multi-token prediction (MTP) head (mtp_num_hidden_layers: 1) that lets the
model self-speculate: it drafts the next token(s) and verifies in one pass, cutting decode
latency without a separate draft model.
That head sits on the acceptance-rate critical path. Quantizing it to 4-bit would push its drafts
away from what the full-precision main model predicts, the verifier would reject more of them, and
the speedup would shrink. So the recipe ignores re:.*mtp.* and keeps the head at bf16 (it is
tiny next to the 27B backbone). The 4-bit backbone does the heavy lifting; the full-precision MTP
head keeps acceptance high. The vision tower (re:.*visual.*) is kept at bf16 for the same
fidelity reason.
Fidelity
Near-lossless versus the bf16 source, ~20 GB vs 55.6 GB bf16 (~36%). GPTQ error compensation and an MSE observer keep the drop from bf16 minimal; the header lists the full characteristics and Quantization covers the recipe.
Quickstart
NVFP4 is auto-detected from config.json (compressed-tensors); no quantization flag needed.--reasoning-parser qwen3 splits the <think> block into reasoning_content.
vllm serve maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 \
--served-model-name fable-fusion-27b-nvfp4 \
--max-model-len 131072 \
--gpu-memory-utilization 0.90 \
--kv-cache-dtype fp8 \
--reasoning-parser qwen3
- Supports up to 262144 tokens; keep at least 128K to preserve thinking quality.
- Add
--language-model-onlyto skip the vision tower and free KV cache for text use.
Speculative decoding (MTP)
The bf16 MTP head enables vLLM's built-in self-speculative decoding:
vllm serve maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 \
--served-model-name fable-fusion-27b-nvfp4 \
--max-model-len 131072 --kv-cache-dtype fp8 --reasoning-parser qwen3 \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'
num_speculative_tokens matches the head depth (mtp_num_hidden_layers: 1). Acceptance is not
penalized by quantization because the head is bf16.
Quantization
| Scheme | NVFP4, W4A4 |
| Weight rounding | GPTQ (Hessian-based error compensation), MSE observer |
| Weights | FP4 (E2M1), group_size=16, tensor_group, FP8 (E4M3) group scales, shared across fused layers |
| Activations | FP4, dynamic per-group, FP8 (E4M3) scales |
| Quantized | all language-model Linear layers |
| Kept in bf16 | MTP head (mtp.*), vision tower (model.visual.*), lm_head |
| Untouched | gated delta-net Conv1d and SSM params (A_log, dt_bias), never Linear |
The model is loaded with AutoModelForImageTextToText so the vision tower is quantized-and-kept
(bf16) rather than dropped; the MTP head, which lives outside the transformers model graph, is
re-attached at bf16 from the base. GPTQ is a quantization-time cost only; inference speed and
format are identical to plain round-to-nearest NVFP4, but it chooses better 4-bit values.
About the base model
DavidAU's Fable-Fusion-711 is a 27B Qwen3.5-family (qwen3_5) vision-language model with thinking
mode, a 256K context window, and an MTP head, already run through Heretic uncensoring.
- 64 decoder layers; hybrid gated delta-net linear attention plus full attention; dense MLP; vision tower.
- MTP head (
mtp_num_hidden_layers: 1) for speculative decoding. - 256K context (
max_position_embeddings262144).
Recommended sampling
Thinking mode is the default.
- Thinking, precise:
temperature=0.6,top_p=0.95,top_k=20 - Thinking, general:
temperature=1.0,top_p=0.95,top_k=20 - Instruct / non-thinking:
temperature=0.7,top_p=0.80,top_k=20
Related
- Base model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP
- Space: Rogue Quants
- Collection: NVFP4 Quants
- Sibling NVFP4 quants:
Notes
- Needs NVIDIA Blackwell (sm_121, e.g. GB10) for accelerated W4A4; pre-Blackwell GPUs run it weight-only.
- The MTP head and vision tower are bf16; enable speculative decoding via
--speculative-config. --reasoning-parseris not auto-detected; pass it explicitly.- No refusal guardrails; you are responsible for how you use it.
License
Apache-2.0, following the base model. Intended use and all responsibility for use follow the base model.
Credits
- Base model: DavidAU
- Uncensoring: Heretic by Philipp Emanuel Weidmann (applied upstream)
- Quantization tooling: llm-compressor / compressed-tensors