base_model: LuffyTheFox/Qwen3.8-27B-Uncensored-Genesis-V1-MTP-GGUF
library_name: gguf
pipeline_tag: text-generation
license: apache-2.0
tags:

• gguf
• nvfp4
• fp4
• imatrix
• blackwell
• llama.cpp
• qwen3.8
• uncensored
• fastmtp
  language:
• en
• zh

Qwen3.8-27B-Uncensored-Genesis-V1 — NVFP4 imatrix GGUF

Absolutely experimental NVFP4 quantisations of
LuffyTheFox/Qwen3.8-27B-Uncensored-Genesis-V1-MTP-GGUF,
calibrated with two different imatrix corpora and built with the
advanced-gguf-quantizer
fork of llama.cpp.

Two calibration variants are provided, each with an MTP and a noMTP build:

All four share the same source, the same per-tensor protection policy, and the
same architecture profile (qwen35dense). They differ only in the calibration
data used to compute the NVFP4 input scales.

Source and architecture

The source is from Q8_K_P, not from BF16 — no BF16 source was
available. As with any lossy-on-lossy requant, tensors that were genuinely q8 in
the source go through one extra rounding step; the substantial number of bf16
source tensors requantise cleanly.

Why imatrix, and why two variants

NVFP4 is a 4-bit float format with per-block input scales. Without an
importance matrix those scales fall back to identity (all 1.0), which works on
Qwen3.x but leaves quality on the table. An imatrix pass measures how much each
weight influences activations on real text, and the quantiser spends the input
scale budget where it matters.

Two calibration corpora were used:

v5 — plain text

Bartowski's v5 calibration set, ~1.7 MB of plain prose. Broad language coverage,
no chat-template structure. Useful as a general-purpose baseline.

v6 — chat-template rendered

Bartowski prose plus 173 conversations (137 base + 36 tool-calling) rendered
through the target model's chat template using
froggeric/Qwen-Fixed-Chat-Templates
v22.5. Rendering preserves the special/control tokens (<|im_start|>,
<|im_end|>, <|tool_call|>, etc.) and role structure the model actually sees
at inference time, so the activation statistics better match the chat/tool-use
distribution. llama-imatrix was run with --parse-special so the rendered
special-token markup is treated as real tokens, not literal text.

If the model is used primarily for chat, reasoning, or tool calling, v6 is the
recommended variant. If it is used for plain completion or broad text work, v5
is a reasonable choice and is slightly cheaper to have produced.

Per-tensor protection policy

Luffy's protection policy was written for the MoE Qwen 3.6 35B (which has
ffn_down_exps). This model is dense, so the equivalent tensor is
ffn_down. Applied that way, every line resolves.

The policy is implemented with anchored --tensor-type regex overrides. The
advanced quantiser matches overrides with std::regex_search and applies them
in reverse order (last match wins), so patterns are anchored with ^...$ to
avoid accidental partial matches (e.g. ssm_a would otherwise also match
ssm_alpha).

Singular-collapse protection (F16)

Four tensors are kept at F16 because they collapse badly under further
quantisation:

A caveat: blk.0.attn_gate / attn_qkv were q8_0 in the source, so F16 here
stores dequantised q8 values. It does the intended job (those tensors keep
q8-level fidelity instead of dropping to 4 bits) but it is not true F16
precision, and it costs more bytes than the source did.

Architecture-specific F32 protection

Additional tensors are kept at F32 for FastMTP loader and Ampere CUDA kernel
compatibility. These protections were not in the original Luffy policy;
they were derived from runtime failures during testing.

Without the 1D-norm F32 protection, the quantiser emits 198 extra one-dimensional
.scale / .input_scale tensors (99 each) that the FastMTP loader at the
investigated revision does not declare, causing a tensor-count mismatch
(expected 1866, got 2064). Keeping the 1D norms at F32 avoids the auxiliary scale
tensors entirely.

Forced NVFP4 (no further collapse)

Catch-all

Everything else eligible takes NVFP4.

Tensor mix (verified in the finished files)

• general.file_type = 39 (NVFP4)
• qwen35.block_count = 65 (64 + 1 MTP)
• qwen35.nextn_predict_layers = 1
• No unexpected one-dimensional norm .scale tensors
• All four F16 protections and all three forced-NVFP4 tensors verified by name

The noMTP derivatives strip the 31 blk.64.* tensors and the
qwen35.nextn_predict_layers metadata key, decrement block_count 65 → 64,
and end at 1835 tensors / 14.46 GB.

FastMTP compatibility

The MTP variants were tested with
HauhauCS FastMTP
using the prebuilt draft sidecar
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf and a patched
llama.cpp build. With the correct local model path (not a Hugging Face cache
variable accidentally pointing at a different quant), the model loaded, the
draft sidecar initialised, generation completed, and speculative decoding
produced measurable acceptance (~96 accepted / 234 generated, acceptance rate
~0.41, mean draft length ~2.23 on the tested prompt).

Important: FastMTP compatibility was verified by actually running the model
in the patched runtime, not by inspecting tensor counts alone. A model that
loads under stock llama.cpp is not automatically FastMTP-compatible.

Running it

llama-cli -m Qwen3.8-27B-Uncensored-Genesis-V1-NVFP4-v2-imx-v6-froggeric.gguf \
  -ngl 99 -c 4096 --temp 0.6 --top-k 20 --top-p 1.0 --min-p 0.0

Sampling above follows the Genesis guidance (temp 0.6, top_k 20, top_p and
min_p disabled). For code, also set --repeat-penalty 1.0: code repeats
identifiers by nature, and any penalty pushes the model off the identifier it
just chose, which is a reliable way to get generated code that won't run.

Give it room to think

Genesis reasons at length before answering. In testing it can spend 900,
even 2600 tokens entirely inside the thinking block without reaching its final
answer. That is the model, not the quantisation — but with a small -n it
looks like the model is broken or rambling. Budget generously (-n 4096 or
more) for anything non-trivial, or you will only ever see it thinking.

FastMTP

llama-server \
  -m Qwen3.8-27B-Uncensored-Genesis-V1-NVFP4-v2-imx-v6-froggeric.gguf \
  --spec-draft-model Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-FastMTP-32K.gguf \
  --spec-draft-ngl all \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 \
  --spec-draft-p-min 0 \
  --ctx-size 32768 \
  --parallel 1 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --n-gpu-layers all \
  --split-mode none \
  --flash-attn on \
  --jinja

Use the MTP file (...v6-froggeric.gguf) for FastMTP, not the noMTP
derivative. The noMTP file is for ordinary inference where the MTP/NextN head is
not wanted.

Hardware notes

• Blackwell (RTX 50xx): native FP4 path, fastest. Build llama.cpp with
  BLACKWELL_NATIVE_FP4 = 1.
• Ampere (RTX 30xx): NVFP4 inference works via fallback kernels. The
  F32 SSM protections in these files prevent a CUDA assert in the patched SSM
  convolution kernel on this architecture.
• The imatrix pass itself runs on Ampere (it is ordinary source-model
  inference); only native NVFP4 CUDA encoding is inefficient on Ampere, so
  quantisation was done CPU-only.

How it was made

v5 imatrix

llama-imatrix \
  -m Qwen3.8-27B-Uncensored-Genesis-V1-Q8_K_P.gguf \
  -f calibration_datav5.txt \
  -o imatrix_v5.dat \
  --output-format dat \
  -ngl all -ts 3,1 -b 2048 -ub 512 -t 6

802 chunks

v6 imatrix

Conversations were rendered through the froggeric v22.5 chat template with
transformers.AutoTokenizer.apply_chat_template(..., tools=tools, add_generation_prompt=False, tokenize=False), concatenated with the prose, and
fed to llama-imatrix with --parse-special:

llama-imatrix \
  -m Qwen3.8-27B-Uncensored-Genesis-V1-Q8_K_P.gguf \
  -f calibration_v6_froggeric_rendered.txt \
  -o imatrix_v6_froggeric.dat \
  --output-format dat \
  -ngl all -ts 3,1 -b 2048 -ub 512 -t 6 \
  --parse-special

708 chunks

Quantisation

Both variants use the same protection policy and the same CPU-only quantise
command, differing only in the imatrix file:

llama-quantize \
  --allow-requantize --mode fast \
  --tensor-type .*=nvfp4 \
  --tensor-type '^blk\..*\.attn_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.post_attention_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_q_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_k_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_norm\.weight$=f32' \
  --tensor-type '^output_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_conv1d\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_dt\.bias$=f32' \
  --tensor-type '^blk\..*\.ssm_a$=f32' \
  --tensor-type '^blk.0.ssm_out.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_gate.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_qkv.weight$=nvfp4' \
  --tensor-type '^blk.0.attn_gate.weight$=f16' \
  --tensor-type '^blk.0.attn_qkv.weight$=f16' \
  --tensor-type '^blk.0.ffn_down.weight$=f16' \
  --tensor-type '^blk.13.ffn_down.weight$=f16' \
  --imatrix imatrix_v6_froggeric.dat \
  Qwen3.8-27B-Uncensored-Genesis-V1-Q8_K_P.gguf \
  Qwen3.8-27B-Uncensored-Genesis-V1-NVFP4-v2-imx-v6-froggeric.gguf \
  Q8_0 6

general.file_type is patched to 39 (LLAMA_FTYPE_MOSTLY_NVFP4) after
quantisation. The catch-all .*=nvfp4 is listed first; more-specific rules
after it. Patterns are anchored to avoid partial matches.

noMTP derivatives

strip_mtp.py removes the 31 blk.64.* tensors and the
qwen35.nextn_predict_layers metadata key, and decrements
qwen35.block_count 65 → 64.

Honest notes

• Speed: these files use the advanced-gguf-quantizer's separate-scale
  NVFP4 format (1866 tensors with explicit .scale / .input_scale tensors),
  not the inline-scale format stock llama.cpp produces (866 tensors). The
  separate-scale format was chosen for FastMTP loader compatibility at the
  investigated llama.cpp revision. It is slightly slower at decode than the
  inline-scale format on otherwise identical hardware, because dequant reads
  scales from separate tensors rather than inline. If FastMTP is not needed,
  an inline-scale build may be faster at the same quality.
• imatrix does not change speed. It only changes the input_scale values
  used during quantisation; the tensor layout and count are the same with and
  without imatrix.
• Quality evidence here is limited to tensor inspection, metadata
  verification, ordinary generation, and FastMTP acceptance measurement. PPL
  and KLD comparisons against the Q8_K_P source and against a data-free NVFP4
  baseline are planned but not yet recorded. Do not treat the imatrix variants
  as proven-better on the basis of this card alone; the calibration rationale is
  sound, but the quality delta should be measured before being claimed.
• The F32 SSM and 1D-norm protections are a runtime compatibility choice,
  not a quality choice. They trade a small amount of size (and a small decode
  cost) for FastMTP loader and Ampere CUDA kernel compatibility. On Blackwell
  with a stock loader they may be unnecessary.

Credits

• Base model and per-tensor protection policy:
  LuffyTheFox
• Genesis V1 finetune: HauhauCS
• Chat template used for v6 calibration:
  froggeric/Qwen-Fixed-Chat-Templates
  v22.5
• FastMTP draft sidecar and patched llama.cpp:
  HauhauCS
• Quantiser: advanced-gguf-quantizer
  (llama.cpp fork)
• Calibration corpus: Bartowski v5 plain text; v6 prose + conversations rendered
  through the target chat template