license: apache-2.0
tags:

• text-generation
• gguf
• nvfp4
• fp4
• qwen3.8
• uncensored
• vision
• multimodal
• mtp
• fast-mtp
• reasoning
• llama-cpp
  language:
• en
• multilingual
  pipeline_tag: text-generation
  base_model:
• HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-GGUF

This repository contains the NVFP4 (NVIDIA 4-bit Floating Point) quantized GGUF release of HauhauCS's Qwen3.8 27B Uncensored Aggressive model.

• Original Creator & Fine-Tuner: HauhauCS
• Quantization Format: NVFP4 (NVIDIA FP4) GGUF
• Architecture: Dense 27B Parameters
• Capabilities: Reasoning / Thinking ON (DeepSeek-style reasoning format), Vision (Multimodal), FastMTP Speculative Decoding

⚡ Highlights & NVFP4 Advantages

• NVFP4 Precision: Extreme throughput and high inference quality with 4-bit floating-point weights, optimized for NVIDIA Blackwell, Ada Lovelace, and Ampere GPUs via llama.cpp NVFP4 CUDA kernels.
• FastMTP Speculative Decoding: Compatible with Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf draft model for massive generation speedups.
• Multimodal (Vision): Includes mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf for visual comprehension and image queries.

🛠️ Build & Run HauhauCS FastMTP

The compact draft uses a standard GGUF d2t token map plus a minimal Qwen3.8 runtime consumer. Build it once.

The example below uses CUDA; for ROCm/HIP or Vulkan, replace -DGGML_CUDA=ON with -DGGML_HIP=ON or -DGGML_VULKAN=ON. For CPU-only, omit the backend flag.

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
git checkout 4df29be4f4c3673f428170fda944a5b19f743bb8

curl -L -o HauhauCS-FastMTP-llama.cpp.patch \
  https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF/resolve/main/HauhauCS-FastMTP-llama.cpp.patch
git apply --check HauhauCS-FastMTP-llama.cpp.patch
git apply HauhauCS-FastMTP-llama.cpp.patch

cmake -S . -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j"$(nproc)"

Note on unpatched builds: If draft loading reports expected 5120, 248320, got 5120, 32768, the FastMTP sidecar is correct but the executable is unpatched. Launch the freshly built ./build/bin/llama-server from this patched checkout.

🚀 Usage Examples (llama-server / llama.cpp)

1. High-Speed Inference with FastMTP Speculative Decoding (Recommended)

Serve the NVFP4 model with the shared FastMTP sidecar:

MODEL=Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4.gguf
DRAFT=Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf
DEPTH=3

CUDA_VISIBLE_DEVICES=0 ./build/bin/llama-server \
  --model "$MODEL" \
  --spec-draft-model "$DRAFT" \
  --spec-draft-ngl all \
  --spec-type draft-mtp \
  --spec-draft-n-max "$DEPTH" \
  --spec-draft-p-min 0 \
  --ctx-size 204800 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --n-gpu-layers all \
  --split-mode none \
  --flash-attn on \
  --no-mmap \
  --temp 1.0 \
  --top-k 20 \
  --top-p 0.95 \
  --min-p 0 \
  --presence-penalty 0 \
  --repeat-penalty 1.0 \
  --jinja \
  --reasoning on \
  --reasoning-effort xhigh \
  --reasoning-preserve \
  --reasoning-format deepseek \
  --host 127.0.0.1 \
  --port 8080

2. Multimodal (Vision) Support

To enable vision support, simply pass the multimodal projector:

  --mmproj mmproj-Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-BF16.gguf

3. Standard Thinking Mode (Without MTP / noMTP)

llama-server \
  --host 0.0.0.0 --port 8080 \
  --model Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-NVFP4-noMTP.gguf \
  --n-gpu-layers all \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --ctx-size 65536 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --temp 0.6 \
  --top-p 0.9 \
  --top-k 64 \
  --min-p 0.05 \
  --repeat-penalty 1.1 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning on \
  --reasoning-effort high \
  --reasoning-preserve \
  --reasoning-format deepseek

📦 Repository Files

Credits & Attribution

All credit belongs to HauhauCS (HuggingFace @HauhauCS).