license: apache-2.0
base_model: DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP
pipeline_tag: text-generation
library_name: gguf
description: "Family of 4 tensor-spliced NVFP4 GGUFs of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP (LOW / MEDIUM / HIGH / VERY-HIGH). Native NVFP4 backbone + per-tier extras (Q5_0/Q8_0/BF16 lm_head, IQ4_XS/Q6_K/BF16 embedding, MTP draft head). MTP speculative decoding. Blackwell sm_120."
tags:

• gguf
• nvfp4
• qwen3.6
• blackwell
• mtp
• speculative-decoding
• llama.cpp
  language:
• en
• multilingual

Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-GGUF

A family of four tensor-spliced hybrid GGUFs of Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP:

• Backbone (all 64 transformer blocks): native NVFP4, converted by us to GGUF from maci0's NVFP4 safetensors checkpoint.
• Extras (output head, token embedding, MTP draft head): quantized per tier, sharing the same NVFP4 backbone across all four files.

The goal was simple: keep the NVFP4 prefill advantage, shrink the file below the 24 GB budget, and get faster decode via a higher-quality MTP draft head. On our dual 16 GB Blackwell setup all tiers fit and run. The MEDIUM/HIGH/VERY-HIGH tiers additionally ship with higher-precision output tensors to reduce the rare repetition-loop failure mode described below. See the notes before treating any numbers as meaningful.

Vision works too. The GGUFs themselves are text-only, but the model is vision-capable: pair it with the vision projector from DavidAU's NEO-MAX-MTP-GGUF repo, unmodified. DavidAU publishes mmproj-BF16.gguf, mmproj-F16.gguf, and mmproj-F32.gguf; we used the BF16 one ourselves (verified byte-identical), but any of the three works. No separate mmproj upload is needed here; just point --mmproj at one of his files.

Follow along & support

I post updates on new conversions, benchmarks, and what I'm working on over on Ko-fi. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, a coffee is always welcome. I do this on consumer hardware and like seeing how far it goes. More is on the way.

☕ ko-fi.com/esatapedico. Updates, work-in-progress, and an optional coffee.

The four tiers

All tiers share the identical 496-tensor NVFP4 backbone (13.70 GB). They differ only in the 10 "extra" tensors (output.weight = LM head, token_embd.weight = embedding, and the 8 MTP draft-head tensors blk.64.*):

Tensor layout per tier (all 1,858 tensors; only the extra-tensor types differ):

The MTP draft head is embedded in the GGUF, so no separate drafter file is needed. Enable it in llama.cpp with --spec-type draft-mtp.

Why these extra tensors? The MTP draft head's job is to predict tokens the main model will accept. Keeping it at IQ4_XS instead of NVFP4 preserves draft quality, which is what makes speculative decoding pay off. The LM head precision is the main quality lever: Q5_0 (LOW) → Q8_0 (MEDIUM) → BF16 (HIGH/VERY-HIGH) progressively remove quantization noise from the exact tensor that determines the next-token distribution.

Anti-loop sampling

We previously recommended the DRY anti-loop sampler for the MEDIUM/HIGH/VERY-HIGH tiers; we no longer do — it interferes with verbatim reproduction of long strings (paths, identifiers, tool arguments), which matters for coding and tool use.

Attribution & provenance

This is a derivative work built entirely from existing Apache-2.0 artifacts. Nothing here was trained or fine-tuned. Credit belongs to:

1. Alibaba / Qwen team for the base model, Qwen/Qwen3.6-27B (Apache-2.0): dense 27B, 64 layers, Gated DeltaNet + Gated Attention hybrid layout, native 262,144-token context.
2. DavidAU for the fine-tune/merge DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP, and for the "LOW" quant design (special low-memory quants with Q5_0/Q6_K output tensor and IQ4_XS extras) from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF.
3. maci0 for the NVFP4 safetensors checkpoint maci0/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4 (Apache-2.0). Note: maci0 ships safetensors; the GGUF conversion was done by us (see "How this was made").
4. This repo's author for the tensor splice itself (combining the sources) and this write-up.

The NVFP4 tensors are native GGML type 40, preserved from the source checkpoint through GGUF conversion with no re-quantization and no dequant-to-requant round trip.

How this was made

1. maci0's NVFP4 safetensors checkpoint was converted to GGUF by us using llama.cpp's convert_hf_to_gguf.py with --outtype auto (which preserves the native NVFP4 tensors as-is instead of dequantizing them). This produced the 19.65 GB all-NVFP4 backbone GGUF (the VERY-HIGH tier, unchanged).
2. Each lower tier was produced with llama-quantize --tensor-type-file <overrides> from that parent, re-quantizing only the 10 extra tensors to the tier's chosen types and copying the NVFP4 backbone verbatim. The per-tier override maps are in this repo (overrides.txt, overrides-medium.txt, overrides-high.txt, overrides-very-high.txt).

No weights were retrained or re-quantized beyond the extra tensors; the NVFP4 backbone is byte-identical in all four files.

Repository contents

• Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-LOW.gguf (15.49 GB)
• Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-MEDIUM.gguf (16.34 GB)
• Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-HIGH.gguf (17.53 GB)
• Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-VERY-HIGH.gguf (19.65 GB)
• overrides.txt (LOW), overrides-medium.txt, overrides-high.txt, overrides-very-high.txt: the per-tensor quantization-type maps from our conversion planning (770 entries each: every weight tensor and its target GGML type, e.g. blk.0.attn_qkv.weight=nvfp4, norms pinned to f32). Useful if you want to reproduce or audit the tensor layout without re-deriving it.

First observations (naive, single-run, not a benchmark)

We did not run a proper benchmark. What follows are informal first impressions from a handful of single-stream runs, included only so others know what to expect. Do not treat these as claims.

• Hardware: 2x NVIDIA Blackwell 16 GB (RTX 5070 Ti + RTX 5060 Ti), split-mode: tensor, llama.cpp via LocalAI, flash attention on, quantized KV cache.
• Prompt: one 180k-token payload (context 204,800). This payload was highly repetitive (it contained the same boilerplate unit 2,000 times), which likely inflates MTP acceptance and speedups; other prompts of different sizes behaved similarly, but this is far from comprehensive.
• Sampling: temperature 0.6, top_p 0.95, top_k 20, min_p 0.
• MTP: --spec-type draft-mtp, spec_n_max 6, spec_p_min 0.75.

The comparison rows:

• maci0 NVFP4 row is our own GGUF conversion of maci0's safetensors checkpoint (the same --outtype auto conversion described above; now the VERY-HIGH tier), not a GGUF published by maci0.
• IQ4XS LOW (DavidAU) row is DavidAU's published GGUF from his NEO-MAX-MTP repo.
• NVFP4-LOW row is the LOW tier (same file that shipped in the previous version of this repo).

Family comparison (naive, one run per tier, 180k payload to ~2k-token essay)

Each tier was measured with the same single 180k-token payload and a fresh LocalAI
process (pod restart) so only one model was resident at a time. All tiers share the
byte-identical NVFP4 backbone; only the 10 extra tensors differ.

Naive first impressions, not a proper benchmark:

• Prefill is effectively identical across tiers (~640 tok/s), which is expected:
  the 496-tensor NVFP4 backbone is byte-identical in all four files.
• Decode rate decreases with tier size (LOW ~19.2, MEDIUM ~18.1, HIGH ~16.8,
  VERY-HIGH ~16.3 tok/s). The higher-precision lm_head/embedding tensors cost decode
  throughput (more bytes per token and slightly lower MTP acceptance).
• No loops observed: every tier finished with finish_reason: stop and
  distinct-5-gram ratio >= 0.98 on the output. Repetition-analysis flags on the
  repetitive-payload runs traced back to the model quoting source sentences verbatim,
  not to degeneration.
• Context used for these runs: the deployed defaults (LOW 307,200 / MEDIUM
  307,200, with 262,144 kept as the documented training context / HIGH 221,184 /
  VERY-HIGH 204,800). The larger
  weights of HIGH/VERY-HIGH do not fit a 307,200-token KV cache on a fresh process
  with the 16 GB per-GPU split, so those tiers run at lower contexts. All tiers
  stopped around 1.7k-2.9k output tokens despite the 20k request, so decode rates
  come from the actually-decoded window.

As with everything above: do not treat these as claims. Single-run, single-hardware,
repetitive payload.

Known caveat: rare non-deterministic repetition loop

During development, one out of five long-generation runs degenerated: the model repeated itself until hitting max_tokens exactly. The signatures were:

• finish_reason: length at exactly the token cap (20,000)
• decode rate far above the typical range seen in our other runs (~38 tok/s where ~20 to 24 was typical)
• MTP acceptance creeping toward 0.98 (repetition is trivially draftable, so it inflates acceptance)

Four subsequent runs, including the exact same 180k to 20k scenario on a fresh process each time, did not reproduce it, and all finished with finish_reason: stop. The cause is not yet fully understood; treat it as a rare, non-deterministic failure mode to watch for, not a guaranteed one. If you see these three signatures together in a long generation, this is likely what is happening.

Mitigation: see "Anti-loop sampling" above. The tier differences remain in the model-side precision of the extra tensors.

Usage

llama.cpp / llama-server

llama-server \
  --model Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-MTP-NVFP4-MEDIUM.gguf \
  --mmproj <optional: Qwen3.6-27B mmproj-BF16.gguf> \
  --ctx-size 204800 \
  --flash-attn on \
  --spec-type draft-mtp \
  --spec-draft-n-max 6 \
  --spec-draft-p-min 0.75 \
  --temp 0.6 --top-p 0.95 --top-k 20

• Requires a recent llama.cpp with NVFP4 (GGML type 40) CUDA kernels and sm_120 support (Blackwell).
• Requires the draft-mtp spec path (merged upstream as LLAMA_CONTEXT_TYPE_MTP).
• For vision input, point --mmproj at one of DavidAU's mmproj-BF16.gguf / mmproj-F16.gguf / mmproj-F32.gguf files from DavidAU/Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF (see the vision note above).
• --spec-draft-p-min 0.75 matters: without it the drafter wastes steps on low-confidence tokens and the speedup shrinks.
• MTP performance is hardware-dependent: try --spec-draft-n-max values 1 through 6 and keep whatever is fastest on your system.

Memory footprint

• Weights: 15.49 / 16.34 / 17.53 / 19.65 GB per tier (all fit comfortably under 24 GB)
• KV cache at 204,800 ctx with q4_0/q4_0: ~3 GB
• Total on a 16+16 GB dual-GPU setup: ~18.5 to 22.7 GB depending on tier, leaving headroom

License

Apache-2.0, identical to every upstream artifact. The base model license governs; GGUF conversion and quantization are transformations, not new training. When redistributing, please retain attribution to Qwen (Alibaba), DavidAU, and maci0 as above.

"Qwen" is a trademark of Alibaba. Trademarks are used here only to identify upstream models; this repository is not affiliated with, sponsored by, or endorsed by Alibaba, DavidAU, or maci0.

Note on this card

This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.