← back to catalog · registered 2026-11-17 14:04

jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-NVFP4-GGUF

jan1k 35B GGUF MoE multimodal second-order
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/jan1k%2FQwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-NVFP4-GGUF"
Response includes
  • classification m-uncensored
  • files 4
  • author_summary 11 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M-U
Primary method

Uncensored (method unknown)

No other method signals detected in this model.
Confidence
LOW
Why this label 3 signals
Weak or ambiguous signals. Best guess based on catalog patterns; treat as tentative and check the evidence below.
  • 'uncensored' in name/tags but no 'abliterated' marker
  • method not identifiable from author declaration alone
  • may be DPO fine-tune, prompt engineering, or unknown technique
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
4K
Likes
3
Model age
4d ago
created 2026-11-17
Downloads over time
Now 3.6K → from 0 ↑ 0%
01.3K2.6K3.9K 0 on Nov 17 3.6K on Nov 22 Nov
Nov 17 → Nov 22 · 6 snapshots · spans 5 days

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Tags
gguf nvfp4 qwen35moe moe mtp multimodal base_model:LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-GGUF base_model:quantized:LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-GGUF license:apache-2.0 endpoints_compatible region:us imatrix

Related

Total size
20.4 GB
Files
4
Quantizations
2
Registered
2026-11-17 14:04
Last updated on HF
2026-11-17 13:35

Files by quantization

F16 1 file 858 MB
mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis-F16.gguf 858 MB ccab14fb download
Auxiliary files 3 files 20.4 GB
Hermes3.6-35B-A3B-Uncensored-Genesis-MTP-NVFP4.gguf 20.4 GB fa9ebb10 download
README.md 3.10 KB 7787cf99 download
.gitattributes 1.66 KB 17e80344 download

README current version from Hugging Face


license: apache-2.0
base_model:

  • LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-GGUF
    tags:
  • gguf
  • nvfp4
  • qwen35moe
  • moe
  • mtp
  • multimodal

Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP NVFP4 GGUF

NVFP4 requantization of LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-MTP-GGUF.

Source: Hermes3.6-35B-A3B-Uncensored-Genesis-MTP-Q8_K_P.gguf (44.5 GB, file_type 7 / Q8_K_P).

Files

File Size Notes
Hermes3.6-35B-A3B-Uncensored-Genesis-MTP-NVFP4.gguf 20.4 GB file_type=39, inline UE4M3 scales
mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis-F16.gguf 0.9 GB F16 vision projector (unmodified from source repo)

Architecture

  • qwen35moe — hybrid MoE: 40 layers (30 DeltaNet/SSM + 10 full-attention, every 4th) + 1 appended MTP/NextN decoder block (blk.40)
  • 256 experts, 8 active + shared expert (~A3B active params)
  • nextn_predict_layers=1 — MTP draft head preserved; works with --spec-type draft-mtp
  • context_length=262144

Quantization recipe

Direct one-step Q8_K_P -> NVFP4 with an advanced llama-quantize build (llama.cpp-derived NVFP4 quantizer):

llama-quantize --allow-requantize --nvfp4-inline-scales-only \
  --tensor-type-file tensor_types.txt <Q8_K_P source> <output> NVFP4 6
  • Inline NVFP4 scales (UE4M3) — compatible with runtimes that do not support separate .scale/.input_scale tensors
  • No imatrix used
  • Per-tensor override list (483 rules): norms/SSM state/router tensors at F32, shared-expert shexp, attn_k/v, ssm_beta, token_embd and singular-collapse-prone blk.0/blk.13 tensors at F16, output.weight at Q6_K
  • The whole MTP block (blk.40) follows the quantizer's default MTP matrix policy: NVFP4

Tensor mix (753 tensors)

Type Count
NVFP4 237
F32 339
F16 175
Q6_K 1 (output.weight)
Q8_0 1 (MTP router)

Verified

Tested with upstream llama.cpp llama-server on Ampere (RTX 3090 / 3080 Ti):

  • Text: coherent reasoning output, arithmetic correct
  • MTP speculative decoding (--spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75): draft acceptance ~93%
  • Vision via the F16 mmproj: image captioning works

Usage

# full GPU offload (needs ~24 GB VRAM incl. mmproj + KV)
llama-server -m Hermes3.6-35B-A3B-Uncensored-Genesis-MTP-NVFP4.gguf \
  --mmproj mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis-F16.gguf \
  -ngl 999 --flash-attn on \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75

# smaller GPUs: keep MoE experts in RAM, dense part on GPU (~11 GB VRAM, ~18 GB RAM)
llama-server -m Hermes3.6-35B-A3B-Uncensored-Genesis-MTP-NVFP4.gguf \
  --mmproj mmproj-Hermes3.6-35B-A3B-Uncensored-Genesis-F16.gguf \
  -ngl 999 --n-cpu-moe 41 -t 8 --flash-attn on \
  --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75

Requires a recent llama.cpp build (or another runtime with qwen35moe + file_type 39 NVFP4 support).

Credits

README history 1 version

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-17 Upload README.md with huggingface_hub a1912e1 3.1 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration