← back to catalog · registered 2026-11-14 21:03

uberkie/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF

uberkie Deepseek 9B GGUF multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/uberkie%2FDeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF"
Response includes
  • classification m3
  • files 5
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
4K
Likes
0
Model age
1w ago
created 2026-11-14
Downloads over time
Now 3.9K → from 0 ↑ 0%
01.4K2.9K4.3K 0 on Nov 14 3.9K on Nov 22 Nov
Nov 14 → Nov 22 · 9 snapshots · spans 8 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja es ru
Quantizations
Q4_K
Tags
gguf llama.cpp lm-studio qwen3_5 qwen35 qwen3.5 hybrid-attention linear-attention reasoning distillation deepseek unsloth

Related

Total size
5.70 GB
Files
5
Quantizations
3
Registered
2026-11-14 21:03
Last updated on HF
2026-11-14 20:11

Files by quantization

Q4_K 1 file 5.70 GB
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf 5.70 GB d32d0258 download
F16 1 file 876 MB
mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf 876 MB f8d09a6d download
Auxiliary files 3 files 26.4 KB
README.md 15.0 KB 9323a0b5 download
LICENSE 11.1 KB d6456956 download
.gitattributes 225 B e2e0016f download

README current version from Hugging Face


base_model:

  • xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
    base_model_relation: quantized
    language:
  • en
  • zh
  • ko
  • ja
  • es
  • ru
    library_name: gguf
    license: apache-2.0
    pipeline_tag: image-text-to-text
    tags:
  • gguf
  • llama.cpp
  • lm-studio
  • qwen3_5
  • qwen35
  • qwen3.5
  • hybrid-attention
  • linear-attention
  • reasoning
  • distillation
  • deepseek
  • unsloth
  • mtp
  • heretic
  • uncensored
  • decensored
  • abliterated
  • 4-bit
  • q4_k_m
  • text-generation
  • image-text-to-text

DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic — GGUF (Q4_K_M)

GGUF conversion of xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic, quantized to Q4_K_M for llama.cpp / LM Studio / Ollama / llama-cpp-python, with a separate F16 vision projector (mmproj) so the multimodal path keeps working.

Base (fp) repo xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
Upstream chain Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B (SFT+RL distill of Qwen3.5-9B) → Heretic ARA ablation → this GGUF
Architecture qwen3_5 (qwen35 in GGUF) — hybrid linear-attention + full-attention, 1 MTP head
Parameters 8.954 B trunk + 0.249 B MTP · 0.456 B (mmproj)
Quantization Q4_K_M trunk (5.02 bpw) + F16 MTP head · 217 Q4_K, 33 Q6_K, 177 F32, 15 F16
Context 262,144 tokens native (rope_theta 10,000,000, interleaved MRoPE [11, 11, 10, 0])
Toolchain llama.cpp build b12182 (fd27b2034)
Modalities text · image · video (vision needs the mmproj file)

📦 Files in this repository

File Size Precision SHA256
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf 5.696 GiB / 6,115,694,112 B Q4_K_M + F16 MTP head d32d0258a08f3ddc1e4e0a2ae31bd9fc7246d3a2657e57296bce6dd164a42c03
mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf 875.6 MiB / 918,167,360 B F16 projector f8d09a6d00ee3318c58f142a62a628346384a878d2f830f470a0483a49690a5e

Both files are required for vision, only the first one is required for text-only use.

Tensor type distribution (main model)

Type Tensors Size
Q4_K 217 3,600.0 MiB
Q6_K 33 1,753.8 MiB
F32 177 4.0 MiB
F16 15 463.9 MiB

The 15 F16 tensors are the MTP head (11 trunk tensors of blk.32 plus blk.32.nextn.{eh_proj,enorm,hnorm,shared_head_norm}), merged in from upstream model-mtp.safetensors without requantizing the trunk. Total: 442 tensors.

Q6_K is applied to the sensitive tensors (attn_v, ffn_down, output-side projections) as defined by the Q4_K_M recipe; all norms, ssm_dt, ssm_conv1d, A_log, dt_bias and the recurrent state tensors stay in F32.


🚀 Quick start

llama.cpp — CLI

# text only
llama-cli -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  -c 8192 -cnv -t 8

# vision (image / video) — both files required
llama-mtmd-cli -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf \
  -c 8192 --image ./photo.jpg

llama.cpp — OpenAI-compatible server

llama-server -m DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf \
  -c 16384 -ngl 99 -t 8 --host 127.0.0.1 --port 8080
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"local","messages":[{"role":"user","content":"Prove that sqrt(2) is irrational."}]}'

The chat template is embedded in the GGUF, so /v1/chat/completions and llama-cli -cnv need no extra flags. Reasoning tokens are returned in reasoning_content; the visible answer arrives after the closing </think> token.

LM Studio

Put both .gguf files in the same folder (a models/<publisher>/<model>/ directory works), then rescan. LM Studio detects the mmproj- prefix automatically and offers the model as a vision model; the Q4_K_M file alone also loads as a text model.

Ollama

# Modelfile (text only — the mmproj projector is not loadable through Ollama today)
FROM ./DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER num_ctx 16384
ollama create deepseek-v4-pro-qwen35-9b-heretic -f Modelfile
ollama run deepseek-v4-pro-qwen35-9b-heretic "Solve: integral of x^2 sin(x) dx"

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf",
    n_ctx=16384,
    n_threads=8,
    verbose=False,
)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "What is 17 * 23? Show your reasoning."}],
    max_tokens=512,
)["choices"][0]["message"]["content"])

🧠 Model architecture

Field Value
GGUF architecture qwen35
Hidden size 4096
Layers 33 (32 transformer + 1 MTP)
Attention layout 3 × linear-attention + 1 × full-attention per group of 4 (full_attention_interval = 4)
Full-attention layers 8 (indices 3, 7, 11, 15, 19, 23, 27, 31)
Q / KV heads 16 / 4 · head_dim 256
FFN SwiGLU, intermediate 12,288
Vocab 248,320 (tokenizer.ggml.pre = qwen35, BPE)
Special tokens BOS: none · EOS: `<
RoPE interleaved MRoPE, sections [11, 11, 10, 0], theta 10,000,000, partial rotary 0.25
SSM (linear attn.) conv kernel 4 · state size 128 · inner size 4096 · groups 16 · time-step rank 32
Tied embeddings No (lm_head present)

Because only 8 of 33 layers hold a growing KV cache, long-context cost is unusually low for a 9B model: roughly 32 KiB per token with an F16 KV cache (≈1 GiB at 32 K, ≈8 GiB at 262 K, before the recurrent state).

MTP head — present as of 2026-11-14

Upstream restored the MTP weights on 2026-11-08 (commit 3add92da, "Restore MTP tensors from the original model") as model-mtp.safetensors (486.6 MB, 15 BF16 tensors). The original Q4_K_M build of this repo predated that commit and therefore carried block_count = 32 with no nextn tensors.

This build merges the MTP head back in. The trunk (all 427 original tensors) is preserved byte-for-byte at its original Q4_K_M/Q6_K/F32 precisions; only the 15 MTP tensors are appended as F16 (bf16 → f16, exponent and mantissa budget identical for these magnitudes, no measurable loss).

Key Before Now
qwen35.block_count 32 33
qwen35.nextn_predict_layers absent 1
qwen35.attention.recurrent_layers 32 entries 33 entries (last two false)
blk.32.* tensors absent 15
tensor count 427 442

The MTP block follows the same layout as the genuine 4B build: blk.32 is a full-attention decoder block, with nextn.eh_proj (4096→8192), nextn.enorm, nextn.hnorm and nextn.shared_head_norm on top. nextn.shared_head_head is intentionally omitted — the loader falls back to model.output, which is exactly what the upstream checkpoint expects (tie_word_embeddings: false).

At load time llama.cpp reports n_layer = 32, n_layer_all = 33 and prints model has unused tensor blk.32.* warnings for the MTP block. This is normal and also happens with the genuine 4B MTP build — those tensors are only consumed when the MTP draft context is built, not during standard decoding.

If you want to regenerate this from scratch instead of merging, convert with the current converter and it will pick up mtp.* automatically:

🖼 Vision encoder (mmproj)

Field Value
Projector type qwen3vl_merger
Depth 27 blocks
Hidden size 1152
Image size / patch 768 / 16
Spatial merge 2
Position embeddings 2304
Tensors 334 (112 × F16, 222 × F32)

The projector is exported at F16 on purpose: quantizing it below F16 measurably hurts OCR and grounding accuracy, and it only costs ~0.9 GiB. If you need it smaller, llama-quantize mmproj-...-f16.gguf mmproj-...-q8_0.gguf Q8_0 gives ~500 MiB and is generally safe for captioning, but not for fine-grained text reading.

When loading Qwen-VL-class models, llama.cpp may warn that grounding tasks want ≥1024 image tokens; add --image-min-tokens 1024 to llama-mtmd-cli / llama-server if you see grounding degradation.


⚙️ How this file was produced

# 1. bf16 safetensors -> F16 GGUF (exact, lossless for these weights)
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic \
  --outtype f16 --outfile model-f16.gguf

# 2. F16 -> Q4_K_M
llama-quantize model-f16.gguf DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-Q4_K_M.gguf Q4_K_M 8

# 3. vision projector, separate file
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic \
  --mmproj --outtype f16 \
  --outfile mmproj-DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-f16.gguf

Conversion ran with transformers 5.x, torch 2.15 (CPU) and llama.cpp b12182. Quantized by Sakura721; the upstream fp weights belong to xtools-at. The converter performs the V-head regrouping required by in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / conv1d / out_proj (32 value heads over 16 key heads) and always writes qwen35.rope.dimension_sections, so the resulting file is self-consistent without patching.


✅ Verification performed on this build

Check Result
Tokenizer round-trip hello → 14556, world → 1814
Metadata integrity 442 tensors · block_count 33 · nextn_predict_layers 1 · ctx 262144 · recurrent_layers 33 entries
Chat template render matches upstream chat_template.jinja (8,736 bytes embedded verbatim)
Text inference coherent reasoning + answer via llama-cli and /v1/chat/completions
Vision inference 384×384 test image described correctly (red border, navy disc, white background)
Server health GET /health → {"status":"ok"}
MTP merge 427/427 original tensors byte-identical: name, dtype, shape, values
MTP weight fidelity merged F16 values match upstream bf16 source bit-for-bit after conversion
MTP detection n_layer 32 / n_layer_all 33, blk.32.* unused-tensor warnings identical to the genuine 4B MTP build

Reference speed on the machine used for conversion — i5-9300H (4C/8T), 16 GB RAM, no GPU offload — with llama-bench:

Test t/s
pp512 12.50
tg128 0.96

That number is CPU-only and not a statement about the model on real hardware; expect several tens of tokens/s for generation on any modern GPU once the file is offloaded with -ngl 99.


🌡 Sampling defaults

A reasonable starting point for this family (adjust to taste):

Parameter Value
temperature 0.7 (0.6 for math/STEM, 0.9 for creative writing)
top_p 0.8
top_k 20
min_p 0.0
repeat_penalty 1.0–1.05

The model is a reasoning model: it emits <think> … </think> before the answer. Keep the full reasoning trace when solving math/STEM problems — truncating it visibly degrades accuracy. If a frontend hides reasoning, make sure it does not also cut the <think> block out of the context.


📊 Inherited benchmark numbers

These results belong to the unquantized base checkpoint and are reproduced here for context only. They were not re-measured on this Q4_K_M file. Quantization error at Q4_K_M on a 9B model is normally small, but treat the numbers as upper bounds.

Benchmark Base model Notes
GSM8K 94.50 % 4-run mean, NVFP4 + vLLM
MMLU-Pro Math 92.40 % MTP-Q8_0 GGUF, 8 K context
MMLU-Pro (Math + Physics + Chemistry) 90.53 % 500 sampled questions per subset, 1,500 total
Gain vs. Qwen3.5-9B base +0.93 pts MMLU-Pro average

Abliteration quality, from the base repo: 3/100 refusals vs. 100/100 for the unmodified model, KL divergence 0.0422. Heretic Arbitrary-Rank Ablation was applied to layers 9–24 with preserve_good_behavior_weight 0.8264, steer_bad_behavior_weight 0.0003, overcorrect_relative_weight 0.9645, neighbor_count 12.


⚠️ Limitations

  • Experimental community model. Hallucinations, arithmetic slips and unstable behaviour on underspecified tasks remain possible; verify high-stakes math, science and factual output independently.
  • Quantization is not free. Q4_K_M introduces a small but real quality drop versus the bf16 weights, most visible in long derivations, rare-script OCR and code. Use Q5_K_M/Q6_K/Q8_0 if you need more headroom.
  • Vision is a bolt-on. The projector is F16 but the language tower is Q4_K_M, so image-conditioned reasoning inherits the same quantization error; small text in images is the first thing to suffer.
  • Reasoning traces can be long. -c must leave room for <think>; running at 2 K context will truncate reasoning mid-flight.
  • Decensored weights. This model has had its refusal behaviour ablated. It will produce content that aligned models filter, including content that is wrong, offensive or unsafe. You are responsible for how you deploy it and for complying with the licences and laws that apply to you.
  • The architecture (qwen3_5) is recent: use a llama.cpp build from b12182 or newer, and avoid forks that predate QWEN35 support — they will refuse to load the file.

📚 Citation

@misc{sakura721_deepseek_v4_pro_qwen35_9b_heretic_gguf,
  title        = {DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic GGUF (Q4\_K\_M)},
  author       = {Sakura721},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Sakura721/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF}},
  note         = {GGUF quantization of xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic}
}

@misc{jackrong_deepseek_v4_pro_qwen35_9b,
  title     = {DeepSeek-V4-Pro-Qwen3.5-9B},
  author    = {Jackrong},
  year      = {2026},
  publisher = {Hugging Face}
}

🙏 Acknowledgements

  • Jackrong — the DeepSeek-V4-Pro-Qwen3.5-9B distillation this line descends from.
  • Heretic — arbitrary-rank ablation tooling used to decensor the base model.
  • Qwen — the Qwen3.5-9B architecture and tokenizer.
  • Unsloth — fine-tuning tooling used upstream.
  • ggml-org / llama.cpp — the converter, quantizer and runtime that made this file possible.

Quantized with llama.cpp b12182 on Windows (CPU-only), from the bf16 safetensors of the base repository.

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-14 Duplicate from Sakura721/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF 9f39abb 14.7 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration