tags:
- mtp
- text-generation-inference
- llama-cpp
- llama-cli
- transformers
- unsloth
- qwen3_5
- tool-use
- heretic
- uncensored
- decensored
- abliterated
- speculative-decoding
- nvidia-blackwell
- rtx-pro-6000
- SM120
- GB202
- B200
- B300
- FP4
- blackwell
- Qwen3.8
- GPTQ
datasets: - HuggingFaceH4/ultrachat_200k
base_model: - DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored
license: apache-2.0
language: - en
pipeline_tag: image-text-to-text
library_name: transformers
Qwen3.8-27B-Ultimate-Heretic-Uncensored-NVFP4-GGUF
Qwen3.8-27B-Ultimate-Heretic-Uncensored-NVFP4 is a high-efficiency NVFP4 quantization of DavidAU/Qwen3.8-27B-Ultimate-Heretic-Uncensored calibrated using GPTQ with an
imatrix_mseweight observer (targetingLinearlayers while preservinglm_headandvisualmodules in full precision). Built upon the multi-stage-tuned Qwen 3.8 27B architecture with DavidAU's Turbo Brilliance V2 50X AGI system and Heretic reasoning and instruct frameworks, this compressed variant integrates 27 reasoning modes, 27 instruct modes, and over 15 dynamic AI constructs to deliver low-footprint, high-throughput inference while maintaining deep refusal removal, multilingual capability, and token efficiency across complex creative, coding, and analytical workloads.
[!NOTE]
MTP — Speculative Decoding: Yes | Accelerated FP4 Tensor Core Performance
[!NOTE]
Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| Qwen3.8-27B-Ultimate-Heretic-Uncensored-NVFP4.gguf | NVFP4 | 19.7 GB | Link | NVFP4 quantized weights. Highly optimized for NVIDIA Blackwell hardware accelerators (e.g., RTX Pro 6000, RTX 50-series, B200, B300), offering native hardware-accelerated FP4 Tensor Core speeds and a strong balance of size and performance. |
| Qwen3.8-27B-Ultimate-Heretic-Uncensored.mmproj.gguf | mmproj | 931 MB | Link | Multimodal projection file. Used for vision/language capabilities. |
NVFP4 Test Run
Tested Qwen3.8-27B-Ultimate-Heretic-Uncensored-NVFP4.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading and MTP speculative decoding.
Performance Metrics
| Metric | Value |
|---|---|
| Average Prompt Processing Speed | 291.66 tokens/s |
| Average Generation Speed | 114.20 tokens/s |
| Total Wall Time | 49.9 seconds |
| Context Length | 16,384 tokens |
| KV Cache (K) | q8_0 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition |
| GPU VRAM Capacity | 96 GB |
| Peak GPU VRAM Usage | 18.33 GiB |
| Peak GPU Utilization | 96% |
| GPU Layers Offloaded | 99 (-ngl 99) |
| Random Seed | 42 |
| Speculative Decoding | MTP (draft_n_max=2) |
| Test Runs | 5 |
Test Shots
| Shot | Test |
|---|---|
| Shot 1 | C: Insert Node at End of Singly Linked List |
| Shot 2 | Jogger and Train Passing Time |
| Shot 3 | Two Pipes Filling a Tank |
| Shot 4 | Students in Examination Rooms |
| Shot 5 | Father and Son Ages |
Run Inference
./build/bin/llama-cli \
-m /path/to/models/Qwen3.8-27B-Ultimate-Heretic-Uncensored-NVFP4.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-ngl 99 \
-c 16384 \
--cache-type-k q8_0 \
--seed 42 \
-n 8192 \
-st \
-p "Your prompt here"
Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.
Snapshots (5)






llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.