base_model:

• xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
  base_model_relation: quantized
  language:
• en
• zh
• ko
• ja
• es
• ru
  library_name: gguf
  license: apache-2.0
  pipeline_tag: image-text-to-text
  tags:
• gguf
• llama.cpp
• lm-studio
• qwen3_5
• qwen35
• qwen3.5
• hybrid-attention
• linear-attention
• reasoning
• distillation
• deepseek
• unsloth
• mtp
• heretic
• uncensored
• decensored
• abliterated
• 4-bit
• q4_k_m
• text-generation
• image-text-to-text

DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic — GGUF (Q4_K_M)

GGUF conversion of xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic, quantized to Q4_K_M for llama.cpp / LM Studio / Ollama / llama-cpp-python, with a separate F16 vision projector (mmproj) so the multimodal path keeps working.

📦 Files in this repository

Both files are required for vision, only the first one is required for text-only use.

Tensor type distribution (main model)

Q6_K is applied to the sensitive tensors (attn_v, ffn_down, output-side projections) as defined by the Q4_K_M recipe; all norms, ssm_dt, ssm_conv1d, A_log, dt_bias and the recurrent state tensors stay in F32.

🚀 Quick start

llama.cpp — CLI

# text only
llama-cli -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  -c 8192 -cnv -t 8

# vision (image / video) — both files required
llama-mtmd-cli -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf \
  -c 8192 --image ./photo.jpg

llama.cpp — OpenAI-compatible server

llama-server -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf \
  -c 16384 -ngl 99 -t 8 --host 127.0.0.1 --port 8080

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"local","messages":[{"role":"user","content":"Prove that sqrt(2) is irrational."}]}'

The chat template is embedded in the GGUF, so /v1/chat/completions and llama-cli -cnv need no extra flags. Reasoning tokens are returned in reasoning_content; the visible answer arrives after the closing </think> token.

LM Studio

Put both .gguf files in the same folder (a models/<publisher>/<model>/ directory works), then rescan. LM Studio detects the mmproj- prefix automatically and offers the model as a vision model; the Q4_K_M file alone also loads as a text model.

Ollama

# Modelfile (text only — the mmproj projector is not loadable through Ollama today)
FROM ./DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER num_ctx 16384

ollama create deepseek-v4-pro-qwen35-9b-heretic -f Modelfile
ollama run deepseek-v4-pro-qwen35-9b-heretic "Solve: integral of x^2 sin(x) dx"

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf",
    n_ctx=16384,
    n_threads=8,
    verbose=False,
)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "What is 17 * 23? Show your reasoning."}],
    max_tokens=512,
)["choices"][0]["message"]["content"])

🧠 Model architecture

Because only 8 of 33 layers hold a growing KV cache, long-context cost is unusually low for a 9B model: roughly 32 KiB per token with an F16 KV cache (≈1 GiB at 32 K, ≈8 GiB at 262 K, before the recurrent state).

The MTP head (layer 32, qwen35.nextn_predict_layers = 1) is preserved in the file but is not exercised by vanilla decoding; it stays available for MTP/NextN-aware builds and for rerunning convert_hf_to_gguf.py --mtp if you want the draft head as a separate file.

🖼 Vision encoder (mmproj)

The projector is exported at F16 on purpose: quantizing it below F16 measurably hurts OCR and grounding accuracy, and it only costs ~0.9 GiB. If you need it smaller, llama-quantize mmproj-...-f16.gguf mmproj-...-q8_0.gguf Q8_0 gives ~500 MiB and is generally safe for captioning, but not for fine-grained text reading.

When loading Qwen-VL-class models, llama.cpp may warn that grounding tasks want ≥1024 image tokens; add --image-min-tokens 1024 to llama-mtmd-cli / llama-server if you see grounding degradation.

⚙️ How this file was produced

# 1. bf16 safetensors -> F16 GGUF (exact, lossless for these weights)
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic \
  --outtype f16 --outfile model-f16.gguf

# 2. F16 -> Q4_K_M
llama-quantize model-f16.gguf DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf Q4_K_M 8

# 3. vision projector, separate file
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic \
  --mmproj --outtype f16 \
  --outfile mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf

Conversion ran with transformers 5.x, torch 2.15 (CPU) and llama.cpp b12182. Quantized by Sakura721; the upstream fp weights belong to xtools-at. The converter performs the V-head regrouping required by in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / conv1d / out_proj (32 value heads over 16 key heads) and always writes qwen35.rope.dimension_sections, so the resulting file is self-consistent without patching.

✅ Verification performed on this build

Reference speed on the machine used for conversion — i5-9300H (4C/8T), 16 GB RAM, no GPU offload — with llama-bench:

That number is CPU-only and not a statement about the model on real hardware; expect several tens of tokens/s for generation on any modern GPU once the file is offloaded with -ngl 99.

🌡 Sampling defaults

A reasonable starting point for this family (adjust to taste):

The model is a reasoning model: it emits <think> … </think> before the answer. Keep the full reasoning trace when solving math/STEM problems — truncating it visibly degrades accuracy. If a frontend hides reasoning, make sure it does not also cut the <think> block out of the context.

📊 Inherited benchmark numbers

These results belong to the unquantized base checkpoint and are reproduced here for context only. They were not re-measured on this Q4_K_M file. Quantization error at Q4_K_M on a 9B model is normally small, but treat the numbers as upper bounds.

Abliteration quality, from the base repo: 3/100 refusals vs. 100/100 for the unmodified model, KL divergence 0.0422. Heretic Arbitrary-Rank Ablation was applied to layers 9–24 with preserve_good_behavior_weight 0.8264, steer_bad_behavior_weight 0.0003, overcorrect_relative_weight 0.9645, neighbor_count 12.

⚠️ Limitations

• Experimental community model. Hallucinations, arithmetic slips and unstable behaviour on underspecified tasks remain possible; verify high-stakes math, science and factual output independently.
• Quantization is not free. Q4_K_M introduces a small but real quality drop versus the bf16 weights, most visible in long derivations, rare-script OCR and code. Use Q5_K_M/Q6_K/Q8_0 if you need more headroom.
• Vision is a bolt-on. The projector is F16 but the language tower is Q4_K_M, so image-conditioned reasoning inherits the same quantization error; small text in images is the first thing to suffer.
• Reasoning traces can be long. -c must leave room for <think>; running at 2 K context will truncate reasoning mid-flight.
• Decensored weights. This model has had its refusal behaviour ablated. It will produce content that aligned models filter, including content that is wrong, offensive or unsafe. You are responsible for how you deploy it and for complying with the licences and laws that apply to you.
• The architecture (qwen3_5) is recent: use a llama.cpp build from b12182 or newer, and avoid forks that predate QWEN35 support — they will refuse to load the file.

📚 Citation

@misc{sakura721_deepseek_v4_pro_qwen35_9b_heretic_gguf,
  title        = {DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic GGUF (Q4\_K\_M)},
  author       = {Sakura721},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Sakura721/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF}},
  note         = {GGUF quantization of xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic}
}

@misc{jackrong_deepseek_v4_pro_qwen35_9b,
  title     = {DeepSeek-V4-Pro-Qwen3.5-9B},
  author    = {Jackrong},
  year      = {2026},
  publisher = {Hugging Face}
}

🙏 Acknowledgements

• Jackrong — the DeepSeek-V4-Pro-Qwen3.5-9B distillation this line descends from.
• Heretic — arbitrary-rank ablation tooling used to decensor the base model.
• Qwen — the Qwen3.5-9B architecture and tokenizer.
• Unsloth — fine-tuning tooling used upstream.
• ggml-org / llama.cpp — the converter, quantizer and runtime that made this file possible.

Quantized with llama.cpp b12182 on Windows (CPU-only), from the bf16 safetensors of the base repository.