base_model: HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive
tags:

• gguf
• quantized
• llama-cpp
• qwen
• qwen3.5
• moe
• vision
• multimodal
• uncensored
  license: apache-2.0
  language:
• en
• zh

Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-GGUF

Full llama.cpp quantization ladder for HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive. K-quants from Q8_0 through Q4 use standard llama-quantize without an importance matrix. Low-bit Q3_K_L / Q3_K_M / Q3_K_S, Q2_K, and all IQ* types use WikiText-2 importance-matrix calibration (200 chunks) when this workspace contains imatrix.dat.

About the Source Model

This repo is a GGUF quantization ladder for HauhauCS/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive: an Aggressive uncensored build based on Qwen/Qwen3.5-35B-A3B (MoE, multimodal, long context). Low-bit K-quants (Q3_K*, Q2_K) and IQ-types use an importance matrix when imatrix.dat was produced in this run—same spirit as our compacted Qwen3.5 GGUF ladder.

For refusal behavior, recommended sampling settings, and mmproj vision tensors, follow the HauhauCS model card and Qwen docs. Note: LM Studio may show 256×2.6B in the params column; HauhauCS reports this is a cosmetic metadata quirk.

Complementary files (read this if a quant is missing here)

The HauhauCS weight index already hosts BF16, Q8_0 through Q6_K, several Q4/Q5 variants, IQ4_XS, IQ3_M, IQ2_M, Q3_K_M, etc. This cahlen companion repo (HF names end with -GGUF) is disk-aware: it adds the extra ladder rungs we use on constrained hardware (e.g. Q5_K_S, Q4_K_S, Q3_K_L / Q3_K_S, Q2_K, IQ3_S, IQ3_XXS, IQ2_S, IQ2_XXS, IQ1_M) with the same WikiText-2 / 200-chunk imatrix workflow as cahlen/qwen3.5-35b-a3b-compacted-GGUF. Pull from HauhauCS if you need a size we do not mirror here.

Available Quantizations

All filenames are prefixed with Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-. The BF16 baseline (65G) was used locally for quantization but is not uploaded to save space; grab it from the HauhauCS source repo if needed. "imatrix" rows used WikiText-2 importance-matrix calibration (200 chunks).

Quality (WikiText-2 Perplexity)

Lower is better. First row is the unquantized baseline.

Measured with llama-perplexity on the WikiText-2 test set (580 chunks, context 512). BF16 baseline evaluated on CPU; quantized variants on NVIDIA RTX 5090.

How to Use

With llama.cpp (text)

llama-cli -m Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_S.gguf --jinja -c 131072 -ngl 99 -p "Hello"

With llama.cpp (vision)

llama-cli -m Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_S.gguf \
  --mmproj mmproj-Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-f16.gguf \
  --jinja -c 131072 -ngl 99

With llama-server

llama-server -m Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_S.gguf --jinja -c 131072 -ngl 99

With Ollama

ollama run hf.co/cahlen/Qwen3.5-35B-A3B-Uncensored-HauhauCS-Aggressive-GGUF:Q4_K_S

With LM Studio

Download any GGUF from the table and load it.

Choosing a Quant

Rough disk size / VRAM guidance (actual usage varies by context length and loader). Quants marked ★ are in this repo; others are on the HauhauCS source repo.

Quantization Details

• Quantized by: cahlen
• Importance matrix: WikiText-2 (wikitext-2-raw-v1, 200 chunks), when generated for this run
• Tool: llama.cpp @ 59d840209
• Hardware: NVIDIA RTX 5090 32GB / Intel Core Ultra 9 285K / 188GB RAM