base_model: LuffyTheFox/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-GGUF
library_name: gguf
pipeline_tag: image-text-to-text
license: apache-2.0
tags:
- gguf
- nvfp4
- fp4
- blackwell
- llama.cpp
- qwen3.5
- deepseek
- uncensored
- mtp
language: - en
- zh
- ko
- ja
- es
- ru
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis — NVFP4 GGUF
NVFP4 quantisation of
LuffyTheFox/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-GGUF,
built with michaelw9999/advanced-gguf-quantizer
(a llama.cpp fork focused on NVFP4/MXFP6 quantization).
Files
| File | Size | Purpose |
|---|---|---|
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-NVFP4.gguf |
5.08 GiB | Language model, NVFP4 (inline UE4M3 scales) |
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-mmproj-F16.gguf |
0.86 GiB | Multimodal projector (vision), unmodified from source |
Source
| Source GGUF | DeepSeek-V4-Pro-Qwen3.5-9B-Uncensored-Genesis-F16.gguf |
| Architecture | qwen35 — hybrid: 24 SSM (gated delta net) + 9 full-attention blocks (interval 4), 33 blocks total |
| Context | 262144 |
| MTP/NextN | nextn_predict_layers = 1 — blk.32 MTP head preserved |
general.file_type |
39 (LLAMA_FTYPE_MOSTLY_NVFP4) |
Requantised directly from the F16 source — no extra rounding step.
Format
Uses --nvfp4-inline-scales-only: native inline UE4M3 scales, no separate.scale/.input_scale tensors. This is the self-contained NVFP4 layout that
works in stock llama.cpp and runtimes like LM Studio that do not implement the
extended NVFP4 scale-tensor contract.
Tensor mix (442 tensors)
| type | count | notes |
|---|---|---|
| NVFP4 | 253 | including token_embd.weight and output.weight |
| F32 | 184 | all 1D norms, ssm_conv1d, ssm_dt.bias, ssm_a, nextn norms |
| F16 | 5 | see below |
F16 protections (singular-tensor collapse + conservative head policy):
blk.0.attn_gate.weight,blk.0.attn_qkv.weightblk.0.ffn_down.weight,blk.13.ffn_down.weightblk.32.nextn.eh_proj.weight(MTP head projection)
blk.0.ssm_out.weight, blk.1.attn_gate.weight, blk.1.attn_qkv.weight are
forced to NVFP4 (they collapse if pushed lower).
Usage
Requires a recent llama.cpp (built ~2026-11 or later; tested at commit1ef8c5d) for qwen35 + NVFP4 support.
llama-server -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-NVFP4.gguf \
-ngl 999 --flash-attn on
Vision (mmproj):
llama-server -m ...-NVFP4.gguf \
--mmproj DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Genesis-mmproj-F16.gguf \
-ngl 999 --flash-attn on
MTP speculative decoding (uses the built-in nextn head):
llama-server -m ...-NVFP4.gguf -ngl 999 --flash-attn on \
--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75
Sampling follows Genesis guidance: temp 0.6, top_k 20, top_p 1.0, min_p 0.0.
Verified
- llama-server on Ampere (RTX 3080 Ti): coherent reasoning output, ~100 tok/s.
- MTP draft head: 36/37 draft tokens accepted (~97%) on a test generation.
- Vision via mmproj: image captioning works.
- CPU inference: coherent output.
Hardware notes
- Blackwell (RTX 50xx): native FP4 path, fastest.
- Ampere (RTX 30xx): works via fallback kernels (verified).
- Quantised CPU-only.
Limitations
No PPL/KLD measurements against the source were performed. Quality verified
only via inference smoke tests.
Credits
- Base model: Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B
- Heretic abliteration: xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
- Genesis processing + source GGUFs: LuffyTheFox
- Per-tensor protection policy: LuffyTheFox (mapped to this dense-hybrid model)
- Quantiser: michaelw9999/advanced-gguf-quantizer