license: apache-2.0
base_model:
- DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF
base_model_relation: quantized
library_name: gguf
pipeline_tag: image-text-to-text
tags: - gguf
- nvfp4
- qwen35
- mtp
- vision
- requantization
language: - en
- ko
Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-NVFP4-GGUF
This is a third-party NVFP4 requantization of DavidAU's Q8_0 GGUF, prepared for Rabi_DELTA.
MTP weights are included. For vision, download the matching mmproj-BF16.gguf from DavidAU
using the link below; the projector is not re-uploaded in this repository.
The original model, fine-tuning, Heretic modifications and Turbo Brilliance work are by
DavidAU. This repository does not claim authorship of those changes.
Source and attribution
- Source repository: DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-GGUF.
- Original base model: DavidAU/Qwen3.8-40B-Ultimate-Heretic-Uncensored.
- Pinned source revision:
16ff9d7e1a9187d0b3d931ec0c34587774748651. - Source file:
Qwen3.8-40B-Ultimate-Heretic-Uncn-Turbo-Brilli-50x-AGI-NEO-MTP-Q8_0.gguf. - Source SHA256:
b63d607f7ce5be9ded6c1cac42d1469ffaa9712dc36289a0f9398a2a30e67b7f. - Upstream declares the Apache-2.0 license. Its model card is preserved in
UPSTREAM_README.md.
The source was already quantized to Q8_0. This release was not quantized directly from
the original BF16 checkpoint. No additional training or model merging was performed.
Quantization and export
Conversion date: 2026-11-18.
- Verify the downloaded Q8_0 GGUF and matching vision projector by SHA256.
- Reconstruct the Q8 weights and quantize 600 main-model projection tensors to NVFP4:
FP4 E2M1 codes, one E4M3 scale per 16 values, and an FP32 tensor scale.
The block scale is selected by an MSE search over the 126 finite positive E4M3 scale words. - Export the existing NVFP4 checkpoint's codes and scales into the standard GGUF NVFP4
super-block layout. There is no additional weight requantization during this export.
Restore GGUF's gated-query row interleaving and linear-attention V-head order. - Store reciprocal native weight divisors as GGUF
.scaletensors; include measured
activation scales as.input_scaletensors. FP32 reciprocal rounding applies. - Preserve all remaining tensors, including Q8_0 vocabulary and MTP matrices, byte-for-byte
from the original Q8 GGUF. They are not all NVFP4. Norms and other original tensor types remain unchanged.
The activation scales were observed on the NVFP4 checkpoint using 128 twelve-token windows
covering Korean, English, Python, JSON/SQL and arithmetic, with 4x amax headroom.
This is a limited calibration set, not a comprehensive quality benchmark.
The tokenizer, original chat template, architecture metadata and MTP metadata are retained.
The model has 96 main layers plus one MTP layer. Metadata retains the upstream 262,144-token
context limit; full-length quality and memory requirements have not been benchmarked.
Files
Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-NVFP4-MTP.gguf: mixed-precision NVFP4 main model, including MTP.export-provenance.json: exact sizes, source revision, per-tensor hashes and conversion details.validation.json: actual GGUF runtime checks and their scope.SHA256SUMS: checksums of the uploaded files; the external mmproj is not included in this list.export_gguf.py: export code for auditing/reproduction; requires the intermediate native artifact and gguf/numpy.LICENSE,NOTICE,UPSTREAM_README.md: license and attribution material.
Vision projector
Use DavidAU's mmproj-BF16.gguf
from the source repository. No custom projector is required. Place it next to the model,
or pass its path using --mmproj.
Expected SHA256: 3f6a3c060ef71e03398749ec11da1429eb9a4638fb1d1430bed537d3a54c6b3d.
Runtime compatibility
Use a recent llama.cpp build supporting Qwen3.5-family architecture, NVFP4 tensors with
per-tensor scales, and Qwen MTP. A GGUF extension alone does not make older runtimes compatible.
Backend activation precision and MTP support can affect numerical results and performance.
Files: model 23,743,746,112 bytes (23.74 GB), vision companion 931,145,888 bytes (0.93 GB).
Example, starting with a modest runtime context and automatic GPU memory fitting:
llama-server \
-m Qwen3.8-40B-Ultimate-Heretic-Uncensored-Turbo-Brilliance-50x-AGI-NEO-NVFP4-MTP.gguf \
--mmproj mmproj-BF16.gguf \
--ctx-size 8192 --gpu-layers auto --fit on \
--spec-type draft-mtp --spec-draft-n-max 4 --port 18080
Flags vary between runtime versions. Fit/offload must be adjusted to available VRAM and RAM.
The NVFP4 KV cache used in the original NInfer deployment is a runtime setting, not data
stored in this GGUF. Do not assume another runtime supports NVFP4 KV or the same memory use.
Estimated performance
User-provided estimate: approximately 55 tokens/s using llama.cpp on an NVIDIA GeForce
RTX 5090 Laptop GPU, with MTP enabled. This figure is an estimate, not a recorded or
controlled GPU benchmark for this GGUF. Actual speed depends on context length, offloading,
runtime version, prompt and MTP acceptance. The CPU-only compatibility tests below are separate.
Validation and limitations
The exporter verifies exact code/scale repacking, independently decodes real NVFP4 blocks
with gguf-py, checks every written tensor's hash after reopening the GGUF, and checks tokenizer,
chat-template and MTP metadata preservation. See validation.json for actual inference tests.
CPU-only smoke tests passed with llama.cpp b11433 / 50569eb87 at a 2,048-token runtime context:
- The model and BF16 vision projector loaded successfully.
- MTP created a draft context and actually proposed/accepted tokens during generation.
17 × 23returned391.- A solid red test image returned
Red.
These short CPU tests establish basic load/inference compatibility. They are not GPU speed,
MTP acceptance-rate, Korean-language or long-context quality benchmarks. GPU performance
of this GGUF has not been measured.
This GGUF has not been evaluated on a standardized quality benchmark. It is a requantization
of an existing Q8 model and may lose quality. Original NInfer speed measurements do not
constitute a benchmark for this GGUF or for llama.cpp. Language tags describe the local validation
focus; the upstream model card lists additional languages.