license: apache-2.0
base_model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:

• gguf
• llama.cpp
• quantized
• imatrix
• vision
• coding
• moe
• experimental

Qwen3.6-35B-A3B Uncensored — karmx mixed-precision GGUF, 128K-tested

GitHub companion repository: KarmSakha/Qwen3.6-35B-A3B-Uncensored-Mixed-128K-GGUF — reproducibility files, pipeline scripts, benchmark evidence, and citation information.

Unofficial, calibration-assisted mixed-precision requantization of
HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
(itself an abliteration of Qwen/Qwen3.6-35B-A3B), packaged by karmx and
validated on a single RTX 5060 Ti 16 GB.

Credit: Qwen team for the base model, HauhauCS for the uncensored variant and
source Q8_K_P weights. This is not an official release of either.

Why this exists

The upstream GGUF ladder jumps from IQ2_M (10.86 GiB, ~2.69 bpw) straight to
Q2_K_P (13.95 GiB). These two files fill the gap with a tensor-specific
mixed recipe: expert down projections (quality-critical) are held at
IQ3_XXS, all attention, SSM output and shared-expert tensors at Q4_K,
embeddings/output at Q5_K, router at Q6_K, and only the bulk expert
gate/up projections drop to the cheapest tier. An imatrix calibrated on a
code/tool/multilingual corpus drives the allocation.

Files

SHA-256 in SHA256SUMS. File size is not VRAM usage; KV cache and
compute buffers add on top.

Measured on RTX 5060 Ti 16 GB (Windows, llama.cpp)

Both files served at 131,072 context fully on-GPU (-ngl 99, Q4_0 KV):

Hybrid architecture (30 gated-deltanet + 10 full-attention layers, 2 KV heads)
keeps the 128K KV cache small and prefill fast. No MTP/nextn block exists in
this base, so there is no speculative-decode head to enable.

Run

hf download karmx/Qwen3.6-35B-A3B-Uncensored-Mixed-128K-GGUF \
  Qwen3.6-35B-A3B-Unc-karmx-mixed-IQ2MQ3-128K-XL.gguf mmproj-f16.gguf \
  --local-dir ./qwen36-unc

llama-server -m qwen36-unc/Qwen3.6-35B-A3B-Unc-karmx-mixed-IQ2MQ3-128K-XL.gguf \
  --mmproj qwen36-unc/mmproj-f16.gguf --alias qwen36-unc \
  -ngl 99 -c 131072 -np 1 -fa on -b 512 -ub 128 -ctk q4_0 -ctv q4_0 \
  --no-context-shift --jinja --host 127.0.0.1 --port 8080

Pick the IQ2Q3 file if you need the extra ~1.3 GiB headroom (bigger KV type,
more slots, or context beyond 131K). Pick XL if you want the extra precision.

Honest limitations

• Requantized from upstream Q8_K_P (--allow-requantize), not the BF16
  safetensors — a small additional quantization error vs a from-source quant.
• imatrix collected on an IQ3_M runner (~3.5 bpw reference), not Q5_K_M.
• Calibration corpus (~131K tokens) was built for a sibling 35B-A3B model;
  same architecture family, but not purpose-built for this exact model.
• Tail-chunk expert coverage measured 99.2–99.6% on the last layers, not 100%.
• Abliterated/uncensored model — few-to-no refusals by design. Use accordingly.
• Perplexity measured on a 12×512-token held-out slice (not a full eval suite);
  XL showed no measurable PPL gain over base — pick it for headroom-of-mind,
  not for proven quality.
• No guarantee on non-Windows runtimes or other GPUs; numbers are from one card.

Held-out perplexity (llama-perplexity, 12 × 512 tokens, CPU, same corpus slice)

Both karmx rungs land slightly below upstream IQ3_M on this slice despite
lower bit-width — consistent with importance-guided allocation spending bits
where they matter. Treat single-corpus PPL as a sanity signal, not a full
capability evaluation.

Recipe

See CALIBRATION-AND-QUANTIZATION.md and
REPRODUCTION.md. Modified-file notice in
MODIFICATIONS.txt; attribution in NOTICE.