license: apache-2.0
base_model: xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
pipeline_tag: text-generation
library_name: gguf
description: "Family of 7 GGUFs of DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic (NVFP4): VERY-LOW through VERY-HIGH sharing a byte-identical 224-tensor NVFP4 backbone, MTP speculative head in every file. Text-only serving; vision tower lives in the checkpoint repo."
tags:
- gguf
- nvfp4
- qwen3_5
- deepseek
- blackwell
- mtp
- speculative-decoding
- llama.cpp
language: - en
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-GGUF
A family of seven GGUF files of DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic (DeepSeek-V4-Pro distill of Qwen3.5-9B: 32-block hybrid, 24 linear-attention + 8 full-attention blocks), converted from esatapedico/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4. The MTP (multi-token prediction) speculative head is baked into every file — no separate drafter needed.
VERY-LOW/COMPACT-LOW/LOW/MEDIUM/MID-HIGH/HIGH/VERY-HIGH— a compact family sharing a byte-identical 224-tensor NVFP4 backbone (96 MLP + 32 full-attention q/k/v/o packed NVFP4 from the checkpoint, plus 96 re-quantized linear-attention gate/qkv/beta/out tensors), differing only in the head tensors (LM head, token embedding, MTP draft head).COMPACT-LOWfills the gap betweenVERY-LOWandLOW.MID-HIGHsits betweenMEDIUMandHIGHwith all three head groups at Q8_0.
The goal: keep native NVFP4 density across the whole model for Blackwell, and offer a size/precision ladder for the tensors that most affect output quality and decode speed.
These GGUFs are text-only on their own. Vision is enabled by pairing any tier with the mmproj-...-f16.gguf projector shipped in this repo (converted from the BF16 source vision tower with convert_hf_to_gguf.py --mmproj, 334 tensors: 112 F16 + 222 F32) via --mmproj.
Follow along & support
I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.
🌐 esatapedico.com. Updates, work-in-progress, and ways to support.
The seven files
| File | Size | SHA-256 | lm_head (output.weight) |
token_embd | MTP head (blk.32) | Attention |
|---|---|---|---|---|---|---|
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-LOW.gguf |
4,768,390,336 bytes | eb3ec611a53cc08f06cc058276730699d8cb00ee0bcc0cb6e3b31ced193a7ae4 |
Q3_K |
Q2_K |
Q2_K |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-COMPACT-LOW.gguf |
5,006,777,536 bytes | a53a225a0d58604328b44151fd36b13b32167b5d86d0986bb6aa1e3da9c107e4 |
Q4_K |
Q3_K |
Q2_K |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-LOW.gguf |
5,286,632,608 bytes | fbfe6ff76af42e3a932d3fd1aea644eb739eae4f4256c41aa05ce2cffd3da4f3 |
Q5_0 |
IQ4_XS |
IQ4_XS |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MEDIUM.gguf |
5,962,063,040 bytes | 4d891dfc7fc4f2d5955b19d8ee5978781e8d6826a6f2fded4abda45774a22f68 |
Q8_0 |
Q6_K |
IQ4_XS |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MID-HIGH.gguf |
6,337,633,472 bytes | 139776d7aae7da029f38fec3bf6d9ca5d952ddc6e15e8978c83e6cefef43d353 |
Q8_0 |
Q8_0 |
Q8_0 |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-HIGH.gguf |
6,915,611,808 bytes | c6ff34cc3b3907a123460cf0192988856e76712230d71eb8ea8d921709816c45 |
BF16 |
Q6_K |
IQ4_XS |
NVFP4 |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-HIGH.gguf |
8,472,796,352 bytes | 5be3d518e442ec04d388a25cd49a0580e23b4bbaca5b83cd290c98c719a2999a |
BF16 |
BF16 |
BF16 |
NVFP4 |
mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf |
918,165,440 bytes | 0a0d985dfb7636a7bf556b18873f05f771e4f838152fe8a4c66d44dde407ebea |
— (vision projector) |
Tensor layout
570 tensors per file: 312 F32 (norms, biases, scales companions) + 130 BF16 (linear-attention SSM path, embeddings where pinned) + 128 checkpoint-native NVFP4 passed through byte-identical from ORIG + 96 re-quantized at tier build (24 blocks × attn_gate, attn_qkv, ssm_beta, ssm_out). The 24 tiny per-layer ssm_alpha vectors stay F32 in every file. Cross-tier backbone identity verified: all 224 NVFP4 tensors pairwise byte-identical across all seven files (SHA-256 per tensor).
Repository contents
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-LOW.gguf(4,768,390,336 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-COMPACT-LOW.gguf(5,006,777,536 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-LOW.gguf(5,286,632,608 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MEDIUM.gguf(5,962,063,040 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MID-HIGH.gguf(6,337,633,472 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-HIGH.gguf(6,915,611,808 bytes)DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-HIGH.gguf(8,472,796,352 bytes)overrides-very-low.txt,overrides-compact-low.txt,overrides-low.txt,overrides-medium.txt,overrides-mid-high.txt,overrides-high.txt,overrides-very-high.txt— per-tensor quantization maps (570 entries each) for reproductionmmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf(918,165,440 bytes) — vision projector, converted from the BF16 source tower
SHA-256
eb3ec611a53cc08f06cc058276730699d8cb00ee0bcc0cb6e3b31ced193a7ae4 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-LOW.gguf
a53a225a0d58604328b44151fd36b13b32167b5d86d0986bb6aa1e3da9c107e4 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-COMPACT-LOW.gguf
fbfe6ff76af42e3a932d3fd1aea644eb739eae4f4256c41aa05ce2cffd3da4f3 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-LOW.gguf
4d891dfc7fc4f2d5955b19d8ee5978781e8d6826a6f2fded4abda45774a22f68 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MEDIUM.gguf
139776d7aae7da029f38fec3bf6d9ca5d952ddc6e15e8978c83e6cefef43d353 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MID-HIGH.gguf
5be3d518e442ec04d388a25cd49a0580e23b4bbaca5b83cd290c98c719a2999a DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-HIGH.gguf
c6ff34cc3b3907a123460cf0192988856e76712230d71eb8ea8d921709816c45 DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-VERY-HIGH.gguf
0a0d985dfb7636a7bf556b18873f05f771e4f838152fe8a4c66d44dde407ebea mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf
Task quality (unified suite, full tier)
GSM8K-200, MMLU-3000 (limit mode, seed 42), IFEval gate subset, ToolCap, LongQA 70k-doc, PPL on diverse text, plus cross-tier KLD anchored on VERY-HIGH (8 KB shared slice). Identical flags every tier, two 16 GB Blackwell cards.
| Tier | Size | PPL | GSM8K | MMLU-3000 | IFEval | ToolCap | LongQA | KLD vs VERY-HIGH |
|---|---|---|---|---|---|---|---|---|
| VERY-LOW | 4.44 GB | 4.5263 | 0.890 | 0.6394 | 30/30 | 100% | 0/3 | 0.0865 |
| COMPACT-LOW | 4.66 GB | 4.4294 | 0.885 | 0.6625 | 30/30 | 100% | 0/3 | 0.0680 |
| LOW | 4.92 GB | 4.4324 | 0.895 | 0.6764 | 30/30 | 100% | 0/3 | 0.0657 |
| MEDIUM | 5.55 GB | 4.4237 | 0.905 | 0.6831 | 30/30 | 100% | 0/3 | 0.0610 |
| MID-HIGH | 5.90 GB | 4.4259 | 0.875 | 0.6809 | 30/30 | 100% | 1/3 | 0.0574 |
| HIGH | 6.44 GB | 4.4243 | 0.895 | 0.6801 | 30/30 | 100% | 0/3 | 0.0609 |
| VERY-HIGH | 7.89 GB | 4.4180 | 0.890 | 0.6723 | 30/30 | 100% | 0/3 | anchor |
Quality saturates at MEDIUM: GSM8K stays in a 0.875–0.905 band across the ladder and MMLU gains are mostly realized by LOW. The KLD ladder is monotonic and an order of magnitude inside the 0.15 similarity band. LongQA 70k-doc retrieval is the weak axis (0–1/3); row-level inspection shows the first question answered correctly but grader-marked wrong, and the other two hit the harness token cap — a harness/model-format question, not a tier-quantization effect (all tiers share the byte-identical backbone).
Speed (llama-bench, dual Blackwell 16 GB)
| Tier | pp512 tok/s | tg128 tok/s |
|---|---|---|
| VERY-LOW | ~6,300–6,800 | ~92–93 |
| COMPACT-LOW | 7,595 | 88.8 |
| LOW | 6,166 | 87.0 |
| MEDIUM | 5,798 | 78.9 |
| MID-HIGH | 6,090 | 80.6 |
| HIGH | 5,785 | 67.8 |
| VERY-HIGH | 6,263 | 67.5 |
Prompt-processing varies run to run (±15% observed across repeats); decode (tg128) falls with size as expected.
Usage
llama.cpp / llama-server
llama-server -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-MTP-MEDIUM.gguf -c 8192 -fa on --jinja -np 1 --port 8080
MEDIUM is the quality/size sweet spot (GSM8K 0.905, MMLU 0.683 at 5.55 GB). MTP speculative decoding is available in every file with a capable backend (-md draft model flags per llama.cpp docs).
How this was made
Self-conversion of the BF16 source to NVFP4 (llm-compressor, datafree W4A16, classic recipe: 128 packed tensors), GGUF conversion with native NVFP4 preserved, then per-tier re-quantization of heads and the linear-attention path with per-tensor override maps (shipped above for reproduction). Full procedure in the build log; every tier gated on tensor census, cross-tier backbone byte-identity, smoke, PPL, speed, and the unified quality suite before release.
Attribution & provenance
Derivative work, Apache-2.0 throughout; nothing here was trained or fine-tuned. Credit:
- xtools-at for the Heretic decensor tune, and Jackrong for the DeepSeek-V4-Pro-Qwen3.5-9B base.
- Red Hat AI / vLLM project for llm-compressor, used for the NVFP4 quantization.
- esatapedico for the conversion, verification, and benchmarking.
License
Apache-2.0, matching the source.
Note on this card
This model card was written by an AI assistant at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (sizes, configs, commands) against the files before relying on it.