license: apache-2.0
language:
- en
base_model: - DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
tags: - gguf
- llama.cpp
- qwen3.8
- mtp
- multi-token-prediction
- bf16
- f16
- q8_0
- quantized
- vision
- mmproj
- uncensored
- heretic
Qwen3.8-40B Ultimate Heretic Uncensored — GGUF (BF16 + Q8_0_L, MTP, mmproj)
GGUF conversions of DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored for use with llama.cpp and compatible runtimes.
This is a format conversion only. No additional training, merging, or editing of weights was done. All model credit belongs to the original authors (see Credits).
Files
| File | Precision | Description |
|---|---|---|
qwen3_40b-MTP-BF16.gguf |
BF16 | Full-precision conversion of the original checkpoint. Best for further quantization or maximum quality. |
qwen3_40b-MTP-Q8_0_L.gguf |
Q8_0 + BF16 key tensors | "Large" Q8_0: most tensors are Q8_0, while the tensors listed below are kept in BF16. |
mmproj-BF16.gguf |
BF16 | Vision projector (multimodal). Required for image input. |
mmproj-F16.gguf |
F16 | Vision projector (multimodal), F16 variant. Use it if your hardware or backend handles F16 better than BF16. |
Use only one mmproj file at a time, together with one of the model files above.
What "Q8_0_L" means. Base type is Q8_0. The following tensors are overridden to BF16 (via --tensor-type-file):
token_embd\.weight=BF16
output\.weight=BF16
.*ffn_down\.weight=BF16
.*ssm_out\.weight=BF16
.*attn_output\.weight=BF16
The result is larger than plain Q8_0 but keeps the embedding, output, FFN-down, SSM-out, and attention-output tensors at full precision. Quality has not been formally benchmarked against plain Q8_0.
Notes
- Vision. The vision projector is provided as
mmproj-BF16.ggufandmmproj-F16.gguf. Load one of them with--mmprojto enable image input. Without anmmprojfile the model works as text-only. - MTP (Multi-Token Prediction). The files are named
MTPbecause the conversion is intended to keep the MTP/NextN head when the source checkpoint contains it. The converter does not add MTP tensors if they are missing from the source. - No importance matrix (imatrix) was used.
Usage
Text-only example with llama-server (sampling values are the ones suggested by the original author):
llama-server \
-m qwen3_40b-MTP-Q8_0_L.gguf \
-c 65536 \
-ngl 99 \
--jinja \
--temp 0.7 --top-k 64 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.0
With vision (image input), add the projector:
llama-server \
-m qwen3_40b-MTP-Q8_0_L.gguf \
--mmproj mmproj-BF16.gguf \
-c 65536 \
-ngl 99 \
--jinja \
--temp 0.7 --top-k 64 --top-p 0.95 --min-p 0.05 --repeat-penalty 1.0
Use --mmproj mmproj-F16.gguf instead if you prefer the F16 projector.
To actually use MTP for faster generation you need a llama.cpp build that supports MTP for this architecture. See the llama.cpp documentation for the current flags.
Suggested settings (from the original model card):
- Temperature 0.5–1.0, repetition penalty 1.0 (off), top-k 64, min-p 0.05, top-p 0.95
- Tool calling: temperature 0.6–0.7, quants Q6/Q8 recommended
- Context: at least 24k tokens, 64k+ for complex or multi-turn use; model maximum is about 256k
- Default reasoning mode is Qwen's
xhigh. Typehelpin a chat, or use{REASON:help}, to open the model's built-in help menu with all reasoning and instruct modes. Details are on the original model card.
Credits
- Original model and fine-tuning: DavidAU — Qwen3.8-40B-Ultimate-Heretic-Uncensored
- Base architecture: Qwen team (Alibaba Cloud)
- GGUF format and tools: llama.cpp
License
Same as the original model: Apache-2.0, per the original model card. Check the original repository and the Qwen license terms before commercial use. This repository is not affiliated with the original authors.
Disclaimer
The source model is described by its author as uncensored (Heretic), so it may produce content that other models refuse. You are responsible for how you use it and for the content it generates. Do not use it to cause harm, and comply with the laws of your country.