tags:
- mtp
- text-generation-inference
- llama-cpp
- transformers
- unsloth
- qwen3_5
- reasoning
- distillation
- deepseek
- sft
- rl
- gspo
- math
- stem
- tool-use
- function-calling
- heretic
- uncensored
- decensored
- abliterated
- speculative-decoding
- nvidia-blackwell
- rtx-pro-6000
- SM120
- GB202
- B200
- B300
- FP4
- blackwell
- Qwen3.5
- GPTQ
library_name: transformers
base_model: - xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
license: apache-2.0
language: - en
pipeline_tag: image-text-to-text
datasets: - HuggingFaceH4/ultrachat_200k
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ-GGUF
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ is a 4-bit NVFP4 GPTQ quantization of DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic, a Heretic v2.0.0.dev0 decensored model based on Jackrong's Qwen3.5-9B reasoning fine-tune distilled from DeepSeek-V4-Pro. The original decensoring reduced refusals from 100/100 to 3/100 at a KL divergence of 0.0422. GPTQ calibration helps preserve model accuracy while reducing memory requirements and improving inference efficiency on NVFP4-capable hardware. Reported original-model benchmarks include 94.50% on GSM8K and 90.53% on a MMLU-Pro STEM sample. These results are not independently verified for this quantized build. The base weights are licensed under Apache 2.0. Experimental outputs require independent verification.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ.gguf | NVFP4 / GPTQ | 8.46 GB | Link | NVFP4/GPTQ quantized weights. Highly optimized for NVIDIA Blackwell hardware accelerators (e.g., RTX Pro 6000, RTX 50-series, B200, B300), offering native hardware-accelerated FP4 Tensor Core speeds. Note: The "UNCENSORED-heretic" designation indicates this model may have modified or removed safety guardrails. |
| DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ.mmproj.gguf | mmproj | 922 MB | Link | Multimodal projection file. Used for vision/language capabilities. |
NVFP4 Test Run
Tested DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading and MTP speculative decoding.
Performance Metrics
| Metric | Value |
|---|---|
| Average Prompt Processing Speed | 280.06 tokens/s |
| Average Generation Speed | 223.05 tokens/s |
| Total Wall Time | 42.0 seconds |
| Context Length | 8,192 tokens |
| KV Cache (K) | q8_0 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition |
| GPU VRAM Capacity | 96 GB |
| Peak GPU VRAM Usage | 28.47 GiB |
| Peak GPU Utilization | 90% |
| GPU Layers Offloaded | 99 (-ngl 99) |
| Random Seed | 42 |
| Speculative Decoding | MTP (draft_n_max=2) |
| Test Runs | 5 |
Test Shots
| Shot | Test |
|---|---|
| Shot 1 | C: Insert Node at End of Singly Linked List |
| Shot 2 | Jogger and Train Passing Time |
| Shot 3 | Two Pipes Filling a Tank |
| Shot 4 | Students in Examination Rooms |
| Shot 5 | Father and Son Ages |
Run Inference
./build/bin/llama-cli \
-m /path/to/models/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-NVFP4-GPTQ.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-ngl 99 \
-c 8192 \
--cache-type-k q8_0 \
--seed 42 \
-n 8192 \
-st \
-p "Your prompt here"
Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.
Snapshots (5)
llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.