license: apache-2.0
base_model:
- Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16
tags: - qwen3.8
- qwen
- 27b
- nvfp4
- gguf
- abliterated
- multimodal
- vision-language
- mtp
- speculative-decoding
- llama.cpp
- blackwell
pipeline_tag: image-text-to-text
library_name: gguf
Qwen3.8-27B Blackfrost Abliterated NVFP4 GGUF (ORIG)
Native-NVFP4 GGUF conversion of Blackfrost-AI/Qwen3.8-27B-ABLITERATED-NVFP4,
the weight-level de-risked (abliterated) Qwen3.8-27B.
This is the ORIG (source-preserving) build: the NVFP4-quantized MLP backbone is
kept natively as GGML type 40, while the DeltaNet/SSM, attention, embeddings, LM
head and MTP head stay in BF16 — the same fidelity strategy as esatapedico's
Qwen3.8-27B-NVFP4-MTP-GGUF ORIG tier, but for the abliterated Blackfrost checkpoint.
Why this file
There was no abliterated NVFP4 GGUF on Hugging Face yet. The official
Blackfrost-AI/Qwen3.8-27B-ABLITERATED-GGUF
ladder is converted from the BF16 master with standard K-quants (Q2_K..Q8_0),
not NVFP4. This repo provides a native NVFP4 option that keeps the MLP at 4-bit
while preserving 16-bit precision where it matters most for long-context quality.
Files
| File | Size | Description |
|---|---|---|
Qwen3.8-27B-Blackfrost-Abliterated-NVFP4-ORIG.gguf |
~29.3 GB | Quality build. NVFP4 MLP + BF16 DeltaNet/attention/embeddings/LM head/MTP. |
Qwen3.8-27B-Blackfrost-Abliterated-NVFP4-VERY-LOW.gguf |
~19.0 GB | Compact build. Everything except norms/embeddings/LM head is NVFP4 (4-bit), including DeltaNet/SSM and attention. |
mmproj-Qwen3.8-27B-Blackfrost-Abliterated-NVFP4-BF16.gguf |
~0.87 GB | BF16 vision projector for image/video input. |
Quantization layout - ORIG vs VERY-LOW
Both files use the same GGML type 40 (NVFP4) for 4-bit weights, but cover different tensor sets:
| GGML type | ORIG | VERY-LOW |
|---|---|---|
NVFP4 |
208 tensors - MLP gate/up/down + selected attention output projections only | 496 tensors - MLP, Gated-Attention q/k/v/o, and DeltaNet/SSM (ssm_alpha/beta/qkv/gate/out) |
BF16 |
298 tensors - DeltaNet, Gated-Attention q/k/v, embeddings, LM head, MTP head | 10 tensors - embeddings, LM head, MTP head |
F32 |
776 tensors - norms, gates, scales | 1352 tensors - norms, gates, scales |
Why it matters: DeltaNet (the SSM/linear-attention path) is a stateful recurrence whose
long-context behavior benefits from higher precision. The ORIG build keeps DeltaNet/attention
at BF16, which is the fidelity-maximizing choice for long-context quality; the VERY-LOW build
compresses the whole network to NVFP4 for the smallest footprint and best VRAM economy. Pick
per your needs: ORIG for maximum long-context fidelity, VERY-LOW for the smallest
size and fastest loading.
Provenance
| Component | Source |
|---|---|
| Base model | Qwen/Qwen3.8-27B (Apache-2.0) |
| Abliteration | Blackfrost-AI/Qwen3.8-27B-ABLITERATED-BF16 (Apache-2.0) |
| NVFP4 quantization | Blackfrost-AI/Qwen3.8-27B-ABLITERATED-NVFP4 (ModelOpt W4A4 NVFP4, Apache-2.0) |
| Conversion | convert_hf_to_gguf.py --outtype auto from llama.cpp master (2026-08-16) |
No retraining or fine-tuning was performed. The conversion preserves the source
NVFP4-packed MLP tensors without a quantization round trip.
Usage
llama.cpp / llama-server (requires NVFP4 + sm_120 Blackwell support)
llama-server \
--model Qwen3.8-27B-Blackfrost-Abliterated-NVFP4-ORIG.gguf \
--mmproj mmproj-Qwen3.8-27B-Blackfrost-Abliterated-NVFP4-BF16.gguf \
--ctx-size 100000 \
--flash-attn on \
--spec-type draft-mtp \
--spec-draft-n-max 4
- Requires a recent llama.cpp with GGML type 40 (NVFP4) CUDA kernels and
sm_120
(Blackwell) support, plus thedraft-mtpspeculative path. - MTP head is embedded in the file (
blk.64.nextn.*), no separate drafter needed. - Note the ORIG build is ~29 GB: fits a 32 GB GPU at reduced context; lower
--ctx-sizeif VRAM is tight.
License
Apache-2.0, matching every upstream artifact. Base model license governs.
Disclaimer
This checkpoint has a deliberately reduced refusal surface (abliterated).
Operators are responsible for their own application-level policy and guardrails.