← back to catalog · registered 2026-11-14 13:03

Sakura721/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-GGUF

Sakura721 Deepseek 4B GGUF multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/Sakura721%2FDeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-GGUF"
Response includes
  • classification m3
  • files 5
  • author_summary 2 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M3
Primary method

Layer-wise ablation

Applied on top of direct removal inherited from the base model.
Confidence
HIGH
Inherited from base model
Why this label 2 signals
Producer identity confirmed by naming conventions, tags or the model card. This label is very unlikely to change.
  • 'heretic' in model name (Heretic-produced)
  • Heretic uses layer-wise optimization (M3) with underlying direction removal (M1)
Refusal direction extracted via
Extraction technique

Difference-of-means

Confidence
HIGH
Why we say so
name contains 'heretic'; Heretic default extraction is difference-of-means (Arditi 2024)
Downloads · 30-day
3K
Likes
0
Model age
1w ago
created 2026-11-14
Downloads over time
Now 3K → from 0 ↑ 0%
01.1K2.2K3.3K 0 on Nov 14 3K on Nov 22 Nov
Nov 14 → Nov 22 · 9 snapshots · spans 8 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en zh ko ja es ru
Quantizations
Q4_K
Tags
gguf llama.cpp lm-studio qwen3_5 qwen35 qwen3.5 hybrid-attention linear-attention reasoning distillation deepseek unsloth

Related

Total size
2.59 GB
Files
5
Quantizations
3
Registered
2026-11-14 13:03
Last updated on HF
2026-11-14 12:52

Files by quantization

Q4_K 1 file 2.59 GB
DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf 2.59 GB 729f8710 download
F16 1 file 641 MB
mmproj-DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-f16.gguf 641 MB 57d73b2c download
Auxiliary files 3 files 24.7 KB
README.md 13.4 KB 8748ab53 download
LICENSE 11.1 KB d6456956 download
.gitattributes 225 B e2e0016f download

README current version from Hugging Face


base_model:

  • xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic
    base_model_relation: quantized
    language:
  • en
  • zh
  • ko
  • ja
  • es
  • ru
    library_name: gguf
    license: apache-2.0
    pipeline_tag: image-text-to-text
    tags:
  • gguf
  • llama.cpp
  • lm-studio
  • qwen3_5
  • qwen35
  • qwen3.5
  • hybrid-attention
  • linear-attention
  • reasoning
  • distillation
  • deepseek
  • unsloth
  • mtp
  • heretic
  • uncensored
  • decensored
  • abliterated
  • 4-bit
  • q4_k_m
  • text-generation
  • image-text-to-text

DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic — GGUF (Q4_K_M)

GGUF conversion of xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic, quantized to Q4_K_M for llama.cpp / LM Studio / Ollama / llama-cpp-python, with a separate F16 vision projector (mmproj) so the multimodal path keeps working.

This is the 4B sibling of the 9B build in Sakura721/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic-GGUF.

Base (fp) repo xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic
Upstream chain Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B (SFT+RL distill of Qwen3.5-4B) → Heretic ARA ablation → this GGUF
Architecture qwen3_5 (qwen35 in GGUF) — hybrid linear-attention + full-attention, 1 MTP head
Parameters 4.326 B (text) · 0.34 B (mmproj)
Quantization Q4_K_M · 5.13 bpw · 222 tensors at Q4_K, 35 at Q6_K, 184 at F32
Context 262,144 tokens native (rope_theta 10,000,000, interleaved MRoPE [11, 11, 10, 0])
Toolchain llama.cpp build b12182 (fd27b2034)
Modalities text · image · video (vision needs the mmproj file)

📦 Files in this repository

File Size Precision SHA256
DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf 2.59 GiB / 2,783,449,920 B Q4_K_M (5.13 bpw) 729f87101cc689add574497ed43ca0a09d80f542643a3fbcfff5a655dfb5b45a
mmproj-DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-f16.gguf 641.3 MiB / 672,424,864 B F16 projector 57d73b2c09f3f9eefc6a125abb0344591f3b21d91d570a6baf95e6aa3172aa92

Both files are required for vision, only the first one is required for text-only use.

Tensor type distribution (main model)

Type Tensors Size
Q4_K 222 1,622.1 MiB
Q6_K 35 1,018.2 MiB
F32 184 3.7 MiB

Q6_K covers the sensitive tensors (attn_v, ffn_down, output-side projections) per the Q4_K_M recipe; all norms, ssm_dt, ssm_conv1d, A_log, dt_bias and recurrent-state tensors stay in F32. Total: 441 tensors, including the 4 MTP tensors of blk.32.


🚀 Quick start

llama.cpp — CLI

# text only
llama-cli -m DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf \
  -c 8192 -cnv -t 8

# vision (image / video) — both files required
llama-mtmd-cli -m DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-f16.gguf \
  -c 8192 --image ./photo.jpg

llama.cpp — OpenAI-compatible server

llama-server -m DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf \
  --mmproj mmproj-DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-f16.gguf \
  -c 16384 -ngl 99 -t 8 --host 127.0.0.1 --port 8080
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"local","messages":[{"role":"user","content":"Prove that sqrt(2) is irrational."}]}'

The chat template is embedded in the GGUF, so /v1/chat/completions and llama-cli -cnv need no extra flags. Reasoning tokens are returned in reasoning_content; the visible answer arrives after the closing </think> token.

LM Studio

Put both .gguf files in the same folder (a models/<publisher>/<model>/ directory works), then rescan. LM Studio detects the mmproj- prefix automatically and offers the model as a vision model; the Q4_K_M file alone also loads as a text model.

Ollama

# Modelfile (text only — the mmproj projector is not loadable through Ollama today)
FROM ./DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER top_k 20
PARAMETER num_ctx 16384
ollama create deepseek-v4-pro-qwen35-4b-heretic -f Modelfile
ollama run deepseek-v4-pro-qwen35-4b-heretic "Solve: integral of x^2 sin(x) dx"

llama-cpp-python

from llama_cpp import Llama

llm = Llama(
    model_path="DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf",
    n_ctx=16384,
    n_threads=8,
    verbose=False,
)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "What is 17 * 23? Show your reasoning."}],
    max_tokens=512,
)["choices"][0]["message"]["content"])

🧠 Model architecture

Field Value
GGUF architecture qwen35
Hidden size 2560
Layers 33 (32 transformer + 1 MTP)
Attention layout 3 × linear-attention + 1 × full-attention per group of 4 (full_attention_interval = 4)
Full-attention layers 8 (indices 3, 7, 11, 15, 19, 23, 27, 31)
Q / KV heads 16 / 4 · head_dim 256
FFN SwiGLU, intermediate 9,216
Vocab 248,320 (tokenizer.ggml.pre = qwen35, BPE)
Special tokens BOS: none · EOS: `<
RoPE interleaved MRoPE, sections [11, 11, 10, 0], theta 10,000,000, partial rotary 0.25
SSM (linear attn.) conv kernel 4 · state size 128 · inner size 2560 · groups 16 · time-step rank 32
Tied embeddings Yes — no separate lm_head; output.weight aliases token_embd.weight

Because only 8 of 33 layers hold a growing KV cache, long-context cost is low for its class: roughly 20 KiB per token with an F16 KV cache (≈0.6 GiB at 32 K, ≈5 GiB at 262 K, before the recurrent state).

MTP head

Unlike the 9B build, this checkpoint genuinely ships an MTP head: model-mtp.safetensors provides 15 mtp.* tensors, which the converter maps to blk.32.nextn.{eh_proj,enorm,hnorm,shared_head_norm} (4 tensors). Metadata is therefore self-consistent out of the box:

  • qwen35.block_count = 33
  • qwen35.nextn_predict_layers = 1
  • qwen35.attention.recurrent_layers = 33 entries, with the last two False

No patching was needed, and the file loads cleanly in llama-server (verified, see below). MTP is not exercised by vanilla decoding but is available to MTP-aware builds for speculative decoding.


🖼 Vision encoder (mmproj)

Field Value
Projector type qwen3vl_merger
Depth 24 blocks
Hidden size 1024
Image size / patch 768 / 16
Spatial merge 2
Position embeddings 2304
Tensors 298

The projector is exported at F16 on purpose: quantizing below F16 measurably hurts OCR and grounding accuracy, and it only costs ~0.64 GiB. If you need it smaller, llama-quantize mmproj-...-f16.gguf mmproj-...-q8_0.gguf Q8_0 gives ~360 MiB and is generally safe for captioning, but not for fine-grained text reading.

When loading Qwen-VL-class models, llama.cpp may warn that grounding tasks want ≥1024 image tokens; add --image-min-tokens 1024 to llama-mtmd-cli / llama-server if you see grounding degradation.


⚙️ How this file was produced

# 1. bf16 safetensors -> F16 GGUF (exact, lossless for these weights)
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic \
  --outtype f16 --outfile model-f16.gguf

# 2. F16 -> Q4_K_M
llama-quantize model-f16.gguf DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-Q4_K_M.gguf Q4_K_M 8

# 3. vision projector, separate file
python convert_hf_to_gguf.py xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic \
  --mmproj --outtype f16 \
  --outfile mmproj-DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-f16.gguf

Conversion ran with transformers 5.x, torch 2.15 (CPU) and llama.cpp b12182. The converter performs the V-head regrouping required by in_proj_qkv / in_proj_z / in_proj_a / in_proj_b / conv1d / out_proj (32 value heads over 16 key heads) and always writes qwen35.rope.dimension_sections, so the file is self-consistent without patching. Quantized by Sakura721; the upstream fp weights belong to xtools-at.


✅ Verification performed on this build

Check Result
Tokenizer integrity 248,320 tokens, 247,587 merges, chat template 9,059 B — byte-identical across conversion and enrichment
Lossless re-packaging 441/441 model tensors and 298/298 mmproj tensors identical in name/dtype/shape
Metadata block_count 33 · nextn_predict_layers 1 · recurrent_layers 33 · 4 × blk.32.nextn.* present
Server load llama-server + --mmproj → loaded multimodal model, GET /health → {"status":"ok"}
Text inference 7*8=? → 56, finish_reason: stop, reasoning trace well-formed
Vision inference 384×384 test image (red frame, white field, navy disc) described correctly over /v1/chat/completions

Reference speed on the machine used for conversion — i5-9300H (4C/8T), 16 GB RAM, CPU only — with llama-cli:

Phase t/s
prompt processing ~22
generation ~7

CPU-only numbers; expect materially higher throughput once the file is offloaded to a GPU with -ngl 99.


🌡 Sampling defaults

A reasonable starting point for this family (adjust to taste):

Parameter Value
temperature 0.7 (0.6 for math/STEM, 0.9 for creative writing)
top_p 0.8
top_k 20
min_p 0.0
repeat_penalty 1.0–1.05

The model is a reasoning model: it emits <think> … </think> before the answer. Keep the full reasoning trace when solving math/STEM problems — truncating it visibly degrades accuracy.


📊 Inherited benchmark numbers

These results belong to the unquantized base checkpoint and are reproduced here for context only. They were not re-measured on this Q4_K_M file. Quantization error at Q4_K_M on a 4B model is normally small but larger than at 9B, so treat the numbers as upper bounds.

Benchmark 4B base 9B sibling Notes
GSM8K 91.77 % 94.50 % 4B: two full runs over all 1,319 test questions (2,638 answers, 2,421 correct), MTP-Q8 + llama.cpp
MMLU-Pro Math 80.40 % 92.40 % 500 sampled questions
MMLU-Pro Physics 74.20 % 89.40 % 500 sampled questions
MMLU-Pro Chemistry 74.80 % 89.80 % 500 sampled questions
MMLU-Pro Average 76.47 % 90.53 % 1,500 questions total

Abliteration quality, from the base repo: 5/100 refusals vs. 99/100 for the unmodified model, KL divergence 0.0378. Heretic Arbitrary-Rank Ablation was applied to the 4B checkpoint before this conversion.


⚠️ Limitations

  • Experimental community model. Hallucinations, arithmetic slips and unstable behaviour on underspecified tasks remain possible; verify high-stakes math, science and factual output independently.
  • Quantization is not free. 5.13 bpw on a 4B model introduces a real, measurable drop versus bf16 — most visible in long derivations, rare-script OCR and code. Use Q5_K_M/Q6_K/Q8_0 if you need more headroom.
  • Vision is a bolt-on. The projector is F16 but the language tower is Q4_K_M, so image-conditioned reasoning inherits the same quantization error; small text in images is the first thing to suffer.
  • Reasoning traces can be long. -c must leave room for <think>; running at 2 K context will truncate reasoning mid-flight.
  • 4B is smaller than 9B. MMLU-Pro gaps of 8–15 points versus the 9B sibling are real; the 4B wins only on footprint and speed.
  • Decensored weights. This model has had its refusal behaviour ablated. It will produce content that aligned models filter, including content that is wrong, offensive or unsafe. You are responsible for how you deploy it and for complying with the licences and laws that apply to you.
  • The architecture (qwen3_5) is recent: use a llama.cpp build from b12182 or newer, and avoid forks that predate QWEN35 support — they will refuse to load the file.

📚 Citation

@misc{sakura721_deepseek_v4_pro_qwen35_4b_heretic_gguf,
  title        = {DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic GGUF (Q4\_K\_M)},
  author       = {Sakura721},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Sakura721/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic-GGUF}},
  note         = {GGUF quantization of xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic}
}

@misc{jackrong_deepseek_v4_pro_qwen35_4b,
  title     = {DeepSeek-V4-Pro-Qwen3.5-4B},
  author    = {Jackrong},
  year      = {2026},
  publisher = {Hugging Face}
}

🙏 Acknowledgements

  • Jackrong — the DeepSeek-V4-Pro-Qwen3.5-4B distillation this line descends from.
  • Heretic — arbitrary-rank ablation tooling used to decensor the base model.
  • Qwen — the Qwen3.5-4B architecture and tokenizer.
  • Unsloth — fine-tuning tooling used upstream.
  • ggml-org / llama.cpp — the converter, quantizer and runtime that made this file possible.

Quantized with llama.cpp b12182 on Windows (CPU-only), from the bf16 safetensors of the base repository.

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-14 Add model card 0882c84 13.4 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration