license: apache-2.0
tags:

• text-generation
• gguf
• nvfp4
• fp4
• moe
• qwen3.6
• hermes
• uncensored
• vision
• multimodal
• genesis
• agentic
• llama-cpp
  language:
• en
• zh
• multilingual
  datasets:
• NousResearch/hermes-function-calling-v1
  pipeline_tag: image-text-to-text
  base_model:
• HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
• LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-GGUF
  base_model_relation: quantized

Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final — NVFP4 GGUF

NVFP4 quantisation of
LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-GGUF,
built with advanced-gguf-quantizer
(a llama.cpp fork focused on NVFP4 quantization).

• Original Model & Genesis Tensor Repair: LuffyTheFox
• Base Model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive (0/465 refusals)
• Hermes Finetune: DJLougen/hermes-qwen3.5-35b-a3b-GGUF (transferred Hermes data onto the uncensored base)
• Architecture: Mixture of Experts (MoE) — 35B total / ~3B active per token (8 routed + 1 shared)
• Multimodal (Vision): Supported via mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis.gguf

Files

v4 — Recommended (inline scales, LM Studio compatible)

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales
without separate .scale/.input_scale tensors. Required for LM Studio /
Pelican and other runtimes that do not support the extended NVFP4 scale
tensor contract.

Imatrix variants will follow after the plain v4 is confirmed working in
target runtimes.

Source

Two-step pipeline: Q8_K_P → F16 intermediate → NVFP4. The F16 step gives
the NVFP4 encoder clean data (no Q8 quantization noise). Quantised CPU-only
(Ampere CUDA NVFP4 encoder hangs on MoE tensors).

Tensor mix (v4)

Tensor protection policy

F16 singular-collapse protection:

F32 architecture-specific protection:

• all norm weights (attn/post_attention/q/k/ssm norms)
• blk.*.ssm_conv1d.weight, blk.*.ssm_dt.bias, blk.*.ssm_a
• blk.*.ffn_gate_inp_shexp.weight (shared expert gate input)
• token_embd.weight → F16

Forced NVFP4 (do not push lower):

Usage

llama-cli -m Hermes3.6-35B-A3B-Uncensored-Genesis-Final-NVFP4-v4.gguf \
  --mmproj mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis.gguf \
  --jinja -c 131072 -ngl 99

• Set K cache and V cache quantization to F16
• GPU offload maximum, active experts 8
• Use chat_template.jinja with --jinja; system prompts included in-repo

Hardware

• Blackwell (RTX 50xx): native FP4 path, fastest
• Ampere (RTX 30xx): NVFP4 inference works via fallback kernels

Reproducibility

# Step 1: Q8_K_P -> F16 intermediate
llama-quantize --allow-requantize \
  Hermes3.6-35B-A3B-Uncensored-Genesis-Final-Q8_K_P.gguf temp_f16.gguf F16 6

# Step 2: F16 -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
  --allow-requantize \
  --nvfp4-inline-scales-only \
  --tensor-type-file tensor_types_protection.txt \
  temp_f16.gguf \
  Hermes3.6-35B-A3B-Uncensored-Genesis-Final-NVFP4-v4.gguf \
  NVFP4 6

Credits

• Genesis algorithm & repair: LuffyTheFox
• Base model: HauhauCS
• Hermes dataset: NousResearch / DJLougen
• NVFP4 quantisation: jan1k
• Quantiser: advanced-gguf-quantizer