license: apache-2.0
tags:

• text-generation
• gguf
• nvfp4
• fp4
• moe
• qwen3.6
• hermes
• uncensored
• vision
• multimodal
• genesis
• llama-cpp
  language:
• en
• multilingual
  pipeline_tag: text-generation
  base_model:
• HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive

Hermes3.6-35B-A3B-Uncensored-Genesis-NVFP4-GGUF

This repository contains the NVFP4 (NVIDIA 4-bit Floating Point) quantized GGUF release of LuffyTheFox's Hermes 3.6 35B A3B Uncensored Genesis model.

• Original Creators & Fine-Tuners: LuffyTheFox & HauhauCS
• Quantization Format: NVFP4 (NVIDIA FP4) GGUF
• Architecture: Mixture of Experts (MoE) — 35B total parameters / ~3B active per token (8 active experts)

⚡ Sampling Recommendations (Genesis SVD Stabilization)

According to tests after Marchenko–Pastur SVD noise-gate stabilization, the model is most stable when Top P and Min P sampling are disabled, as the tensor noise gate has been removed. Only Temperature and Top K sampling should be used.

Recommended Parameters:

🛠️ Tool Calling & Hermes Agent Compatibility Fix

If running with the qwen3.6-froggeric chat template through Hermes Agent, the template defaults to XML tool call output (<function=...><parameter=...>), whereas Hermes Agent expects standard OpenAI JSON tool calls.

To ensure seamless JSON tool calling, pass tool_call_format: "json" in --chat-template-kwargs:

{"enable_thinking": false, "tool_call_format": "json"}

(Or set line 2 of chat_template.jinja default from xml to json).

💡 Recommended System Prompts

1. Default / First Experience (High Stability)

You are Qwen (Tongyi Qianwen), a large language model developed by Alibaba Group's Tongyi Lab.

2. Agentic Tasks (JSON Schema Output)

You are Qwen (Tongyi Qianwen), a large language model developed by Alibaba Group's Tongyi Lab. You are a helpful assistant that answers in JSON. Here's the json schema you must adhere to:
<schema>
{schema}
</schema>

🚀 Usage Example (llama-server / llama.cpp)

1. Coding / Precise Tasks (64K Context)

llama-server \
  --host 0.0.0.0 --port 8080 \
  --model Hermes3.6-35B-A3B-Uncensored-Genesis-NVFP4.gguf \
  --override-kv qwen2moe.expert_used_count=int:8 \
  --n-gpu-layers all \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --ctx-size 65536 \
  --batch-size 2048 \
  --ubatch-size 512 \
  --temp 0.6 \
  --top-p 1.0 \
  --top-k 20 \
  --min-p 0.0 \
  --seed 42 \
  --presence-penalty 0 \
  --repeat-penalty 1.0 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning off \
  --chat-template-kwargs '{"enable_thinking":false, "tool_call_format":"json"}'

2. Multimodal (Vision) Support

Simply add the mmproj projector parameter:

  --mmproj mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis-F16.gguf

📦 Repository Files

Credits & Attribution

• LuffyTheFox for the Genesis post-training calibrations and Hermes fine-tuning.
• HauhauCS for the base fine-tunes.
• Chat template by froggeric & qweefchief.

good luck on your journey, YMMV!