tags:
- mtp
- text-generation-inference
- llama-cpp
- Qwen3.5
- flash
- math
- reasoning
- sft
- deepseek-v4
- bf16
- v3.0
- abliterated
- heretic
- uncensored
- decensored
- instruction-following
license: apache-2.0
base_model: - prithivMLmods/Qwen3.5-9B-DS-v4-Flash-uncensored-heretic
language: - en
pipeline_tag: image-text-to-text
library_name: transformers

Qwen3.5-9B-DS-v4-Flash-uncensored-heretic-GGUF
Qwen3.5-9B-DS-v4-Flash-uncensored-heretic is a fine-tuned (trained on DeepSeek V4 Flash reasoning traces) and uncensored version of Qwen/Qwen3.5-9B, created using Heretic v2.0.0.dev0. The model combines reasoning-oriented fine-tuning with Heretic-based uncensoring to explore improvements in long-form reasoning, mathematical problem solving, scientific analysis, coding, and instruction following while reducing refusal behavior.
[!NOTE]
MTP — Speculative Decoding: Yes | This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios. Its uncensored behavior does not guarantee factual accuracy, and outputs should be independently evaluated.
[!NOTE]
Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.BF16.gguf | BF16 | 18.4 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q3_K_L.gguf | Q3_K_L | 5.15 GB | Link | Lower quality but usable, good for low RAM availability. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q3_K_M.gguf | Q3_K_M | 4.8 GB | Link | Low quality, smallest file size. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q4_K_M.gguf | Q4_K_M | 5.83 GB | Link | Good quality, default size for most use cases, recommended. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q4_K_S.gguf | Q4_K_S | 5.49 GB | Link | Slightly lower quality with more space savings, recommended. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q5_K_M.gguf | Q5_K_M | 6.67 GB | Link | High quality, recommended. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q5_K_S.gguf | Q5_K_S | 6.47 GB | Link | High quality with slight space savings, recommended. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q6_K.gguf | Q6_K | 7.56 GB | Link | Very high quality, near perfect, recommended. |
| Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.mmproj-bf16.gguf | mmproj-bf16 | 922 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |
Q5_K_M Test Run
Tested Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q5_K_M.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading and MTP speculative decoding.
Performance Metrics
| Metric | Value |
|---|---|
| Average Prompt Processing Speed | 515.15 tokens/s |
| Average Generation Speed | 269.61 tokens/s |
| Total Wall Time | 25.8 seconds |
| Context Length | 16,384 tokens |
| KV Cache (K) | q8_0 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition |
| GPU VRAM Capacity | 96 GB |
| Peak GPU VRAM Usage | 7.05 GiB |
| Peak GPU Utilization | 84% |
| GPU Layers Offloaded | 99 (-ngl 99) |
| Random Seed | 42 |
| Speculative Decoding | MTP (draft_n_max=2) |
| Test Runs | 5 |
Test Shots
| Shot | Test |
|---|---|
| Shot 1 | C: Insert Node at End of Singly Linked List |
| Shot 2 | Jogger and Train Passing Time |
| Shot 3 | Two Pipes Filling a Tank |
| Shot 4 | Students in Examination Rooms |
| Shot 5 | Father and Son Ages |
Run Inference
./build/bin/llama-cli \
-m /path/to/models/Qwen3.5-9B-DS-v4-Flash-uncensored-heretic.Q5_K_M.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-ngl 99 \
-c 16384 \
--cache-type-k q8_0 \
--seed 42 \
-n 8192 \
-st \
-p "Your prompt here"
Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.
Snapshots (5)






llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp(v1.0.0) by prithivMLmods.