Qwen3.8-27B-TURBO-Fable-ColdFusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-NVFP4-GGUF

A mixed-precision GGUF build of the Qwen3.8-27B-TURBO Fable-ColdFusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP model, requantized from Q8_0 using NVIDIA's native 4-bit floating-point format (NVFP4) for the bulk of the model with F32 preservation for critical normalization and MTP tensors.

Model Architecture

• Base Model: Qwen3.8-27B-TURBO by the Qwen team
• Architecture: Qwen3.8 (qwen35)
• Parameters: 27.3B
• Embedding Length: 5,120
• Context Length: 262,144 tokens
• Blocks: 65 (64 transformer + 1 MTP head)
• Attention Heads: 24 (4 KV heads)
• Rope Base: 10,000,000
• SSM Group Count: 16
• SSM State Size: 128

Quantization Scheme

This model uses a carefully crafted mixed-precision quantization approach:

• NVFP4 (502 tensors): Attention projections (Q/K/V), FFN layers, SSM kernels, and the MTP head projection (eh_proj). NVFP4 is NVIDIA's native 4-bit floating-point format optimized for Blackwell GPUs.
• F16 (4 tensors): Early layer attention QKV and FFN-down weights (blk.0.attn_qkv, blk.0.ffn_down, blk.13.ffn_down, blk.0.attn_gate) — protected from the severe singular collapse that affects Q4/NVFP4 at this scale.
• F32 (360 tensors): Layer normalization weights, SSM state parameters, token/output embeddings, rope factors, and MTP normalization tensors.

Original Q8_0 size: 29 GB
NVFP4/F32/F16 size: ~15 GB (49% reduction)

MTP Preservation

The MTP head tensors are preserved at full precision to maintain multi-token prediction support:

• blk.64.nextn.enorm.weight (F32)
• blk.64.nextn.hnorm.weight (F32)
• blk.64.nextn.shared_head_norm.weight (F32)
• blk.64.ssm_conv1d.weight (F32)
• blk.64.ssm_dt.bias (F32)
• blk.64.ssm_a (F32)

The qwen35.nextn_predict_layers metadata key (value=1) is retained for GGUF loader compatibility. Note: qwen3.8 MTP uses a different head structure than qwen3.5, so Ollama's MTP support for qwen3.8 is still being developed.

Files

Usage

Ollama

Create a Modelfile:

FROM Qwen3.8-27B-TurboFCFusion-735-882-NVFP4-MTP.gguf
FROM mmproj-F16.gguf

PARAMETER num_ctx 32768
PARAMETER num_predict 4096

TEMPLATE """{{- if .System }}
<|system|>
{{ .System }}
<|end|>
{{- end }}
{{- if .Prompt }}
<|user|>
{{ .Prompt }}
<|end|>
{{- end }}
<|assistant|>
{{ .Response }}
<|end|>"""

Then:

ollama create qwen3.8-27b-nvfp4 -f Modelfile
ollama run qwen3.8-27b-nvfp4 "Your prompt here"

llama.cpp / llama-server

./llama-server -m Qwen3.8-27B-TurboFCFusion-735-882-NVFP4-MTP.gguf -c 2048 -t 8 -ngl 999

Performance

NVFP4 provides significantly faster inference on Blackwell GPUs (RTX 5060 Ti / 5070 / 5080 / 5090) compared to traditional Q4_K_S or Q8_0 formats, while maintaining quality on par with higher-precision variants for practical use. The mixed-precision approach protects the most sensitive tensors from the severe singular collapse that affects Q4 at this model size.

Build Details

• Built with: llama.cpp (v0.3.0-dev, build 10645)
• Quantization command: llama-quantize with custom tensor-type file specifying NVFP4/F32/F16 distribution
• Calibration: Code-heavy text corpus (C, Python, Rust, JavaScript) generated from GitHub open-source repositories
• Base model source: DavidAU's Q8_0 GGUF build

Credits

• Original model: Qwen team — Qwen3.8-27B-TURBO architecture and base weights
• GGUF conversion: DavidAU — Q8_0 GGUF build and finetune work
• NVFP4 requantization: AIconjured — this build

License

See the original model license at Qwen/Qwen3.8-27B.