base_model: xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic
base_model_relation: quantized
library_name: exllamav3
license: apache-2.0
pipeline_tag: text-generation
tags:
- exl3
- exllamav3
- quantized
- qwen3_5
DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic — EXL3 5 bpw
EXL3 quantization of xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic at revision e4bfa83c72896cbb4990c909967d6f27bb97bac4. Original weights without a LoRA adapter; the upstream tokenizer and chat template are preserved.
Quantization
| Setting | Value |
|---|---|
| Main linear weights | 5 bits per weight target |
| Output head | 6 bits |
| Vision tower | 6 bits |
| MTP layers | 4 bits |
| Token embeddings | BF16 (16 bits) |
| ExLlamaV3 converter | 1.6.6 |
| Codebook | mul1 |
| Calibration | Bundled corpus, 250 rows × 2048 tokens |
| Output scales | auto |
| Weight files | 7,826,135,525 bytes |
The target bitrate applies to the main linear weights. Embeddings, the head, vision/MTP components, scales and metadata have their own storage, so the total file size is higher than a uniform 5-bit estimate.
Use
Load this folder with ExLlamaV3 1.6.6 or a compatible ExLlamaV3 backend such as TabbyAPI. Use the included original chat template. KV cache precision is a runtime setting; Q8 was used for local text-generation checks. Quantization does not change the architecture's configured context limit. Memory use depends on context length and which components are loaded.
See conversion.json for the source pin, tensor checksums and reproducible converter arguments. smoke-test.json, if present, records short deterministic text-generation checks and their context/cache settings; these are not a coding benchmark or a validation of the full context window, vision tower or MTP.
Attribution and license
Upstream model: xtools-at/DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic. Upstream ancestry includes Jackrong/DeepSeek-V4-Pro-Qwen3.5-9B. The source model card declares Apache-2.0; that license is included in LICENSE.
The upstream model card is preserved in UPSTREAM_README.md. Its training, distillation and performance statements are upstream claims, not independently established by this conversion.