← back to catalog · registered 2026-11-16 21:03

Ermzzz999/Qwen3.6-35B-A3B-GSQ-2bit-abliterated-hybrid-GGUF

Ermzzz999 Qwen 35B GGUF MoE
Your rig guess connected
? Why do I need an app?
Reading your rig…

This is a rough estimate. Install the free app - we'll show exact numbers.

Reading real hardware from your app right now. Numbers below are exact.

Below is the per-quantization compatibility for this model.

curl -H "Authorization: Bearer $ABL_KEY" \
     "https://abliteration.org/api/v1/models/Ermzzz999%2FQwen3.6-35B-A3B-GSQ-2bit-abliterated-hybrid-GGUF"
Response includes
  • classification m8
  • files 3
  • benchmarks 11 entries
  • author_summary 1 models
  • readme_text full
10 credits · hourly refresh · ~4 KB payload Get an API key →
Abliteration classifier · v1.0.0
M8
Primary method

Repackaging (quantization)

Applied on top of direct removal inherited from the base model.
Confidence
MEDIUM
Inherited from base model
Why this label 3 signals
Method inferred from partial signals - repository name, related files, or tag patterns. Producer identity not confirmed; label may sharpen or shift as we gather more evidence.
  • 'abliterated' in name/tags
  • is_gguf=1
  • assume M1 (base ablation) + M8 (GGUF quant) - default when producer unknown
Refusal direction extraction

No specific extraction method could be identified for this model. The producer either did not document it or used a proprietary pipeline.

What is a refusal direction? →
Downloads · 30-day
3K
Likes
0
Model age
5d ago
created 2026-11-16
Downloads over time
Now 2.8K → from 0 ↑ 0%
01K2K3.1K 0 on Nov 16 2.8K on Nov 22 Nov
Nov 16 → Nov 22 · 7 snapshots · spans 6 days

Benchmarks

Portrait before abliteration
Benchmarks of the base model as it stood before the refusal-removal operation. Compare with the numbers above to see what the operation cost.
Benchmark Score Source
Entertainment 1.4 UGI
Hazardous 0 UGI
Natural Intelligence 25.43 UGI
Political lean -19.6% UGI
Sensitive-Info 14.03 UGI
SocPol 2.6 UGI
UGI 16.02 UGI
Willingness (10) 2 UGI
W10-Adherence 0 UGI
W10-Direct 4 UGI
Writing 35.83 UGI

Genealogy 0 direct forks

Full fork graph →

This model's place in the market. Above: what it was derived from. Below: the tree of everything derived from it.

Metadata

License
apache-2.0
Languages
en it
Tags
gguf llama.cpp moe abliterated uncensored quantization 2-bit text-generation en it arxiv:2604.18556 base_model:ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ

Related

Total size
10.8 GB
Files
3
Quantizations
1
Registered
2026-11-16 21:03
Last updated on HF
2026-11-16 20:51

Files by quantization

Auxiliary files 3 files 10.8 GB
Qwen3.6-35B-A3B-GSQ-2bit-abliterated-2.672bpw.gguf 10.8 GB b10ae67d download
README.md 6.34 KB c96760f4 download
.gitattributes 1.57 KB 5f4f93d9 download

README current version from Hugging Face


language:

  • en
  • it
    license: apache-2.0
    base_model:
  • Qwen/Qwen3.6-35B-A3B
  • ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ
  • huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated
    tags:
  • gguf
  • llama.cpp
  • moe
  • abliterated
  • uncensored
  • quantization
  • 2-bit
    library_name: gguf
    pipeline_tag: text-generation
    arxiv: 2604.18556

Qwen3.6-35B-A3B — GSQ 2-bit experts + abliterated trunk

Hybrid 35B-A3B MoE: the 2-bit GSQ expert weights kept bit-exact, wrapped in huihui's abliterated trunk,
with the expert down-projections abliterated per layer. 11.58 GB, 2.67 bits per weight, vanilla llama.cpp.

Experimental model. It started as an attempt to build a Qwen3.6-35B-A3B from what I consider the best
2-bit weights available — ISTA-DASLab's GSQ quantization
(ISTA-DASLab/Qwen3.6-35B-A3B-2Bit-GSQ) —
but in abliterated form, taking the abliterated bits from huihui-ai
(huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated).
The result is a hybrid: the GSQ experts kept at 2 bits, huihui's abliterated trunk, and a per-layer
abliteration of the expert down-projections.

Unofficial derivative — not affiliated with Alibaba/Qwen, ISTA-DASLab or huihui-ai.
Uncensored model. Use responsibly and legally.

At a glance

File 11.59 GB · 34.66B params · 2.672 bpw
Experts gate/up 2.25 bpw, Q2_0 — lossless transcode of the GSQ values
Experts down 2.62 bpw, Q2_K
Trunk Q5_K, imatrix-guided
Embeddings / output Q8_0 / Q6_K
Router F32
Context 262,144
Speed, M3 Pro 42-45 tok/s decode · 225 tok/s prefill
Perplexity (wikitext-2, 40×512) 6.9247 ±0.16
MTP block pruned

Weight distribution

Tensors Count Type bpw Size % of file
ffn_gate_exps 40 Q2_0 2.25 3.020 GB 26.1%
ffn_up_exps 40 Q2_0 2.25 3.020 GB 26.1%
ffn_down_exps 40 Q2_K 2.62 3.523 GB 30.4%
token_embd 1 Q8_0 8.50 0.540 GB 4.7%
output 1 Q6_K 6.56 0.417 GB 3.6%
attn_qkv 30 Q5_K 5.50 0.346 GB 3.0%
attn_gate, ssm_out 30 + 30 Q5_K 5.50 0.173 + 0.173 GB 3.0%
attn_q, attn_output 10 + 10 Q5_K 5.50 0.115 + 0.058 GB 1.5%
ffn_gate_inp (router) 40 F32 32.0 0.084 GB 0.7%
ffn_{gate,up,down}_shexp 40 each Q5_K 5.50 0.029 GB each 0.6%
norms, attn_k/v, ssm_alpha/beta 51 F32 / Q5_K — 0.060 GB 0.5%

By provenance:

Source Parameters Bytes
ISTA-DASLab GSQ 2-bit — exact (gate/up) 62.0% 52.1%
ISTA-DASLab GSQ 2-bit — abliterated, re-encoded (down) 31.0% 30.4%
huihui-ai abliterated (trunk, embd, output, router) 7.1% 17.4%

Build steps

Step What Result
1 GSQ gate/up experts → Q2_0, verified block-by-block against source byte-identical packing, 2.25 bpw
2 Expert down_proj abliteration: rank-1, per layer, direction from donor Gram 39/40 layers, ‖ΔW‖/‖W‖ = 1.7%, row norms preserved
3 Abliterated down → Q2_K 2.62 bpw
4 Trunk quantized with imatrix (510 tensors, 76 chunks) Q5_K attn/SSM/shared, Q6_K output, Q8_0 embd, F32 router
5 MTP block pruned after the per-layer KV fix (block count, recurrent array) 733 tensors, block_count 40

Files

File Size SHA-256
Qwen3.6-35B-A3B-GSQ-2bit-abliterated-2.672bpw.gguf 11.59 GB b10ae67d02f4e80ba0cef862006925814cb436e6eb02407dd26b257bdb59831c

Results

Perplexity, wikitext-2, 40 chunks × 512, one harness:

Build Size PPL
This file 11.59 GB 6.9247
Same pipeline, Q8_0 trunk 13.46 GB 6.9293
Same source, gate/up 4.5 bpw + down 5.56 bpw 21.71 GB 6.9906
Reference: public Q3_K from true BF16 weights 17.17 GB 6.1898

The source is already 2-bit: extra precision preserves it, it does not add information. The last row is what
true BF16 weights buy (≈0.73 PPL) — the cost of the 2-bit source, not of this file.

Battery (single sample, thinking off): probe correct, Italian prose coherent, code runs, no repetition,
compliant on a safety probe the base model refuses.

Limits

  • Down-projections carry the only meaningful re-quantization error (no 4-level grid after abliteration).
  • Abliteration is spot-checked; no refusal benchmark; thinking-off only. Thinking is ON by default upstream.
  • PPL: one corpus, one context length, one harness — relative, not absolute.
  • Text-only. No vision tower. MTP pruned.

Requirements

  • GGUF runtime with Q2_0 (ggml type 42) — llama.cpp builds from late 2026.
  • No patched build needed to run it. Do not re-quantize with stock llama-quantize.
  • llama-server -m <file> -c 16384 --flash-attn -ngl 99

License

Apache-2.0, with attribution to Qwen (architecture, tokenizer), ISTA-DASLab (GSQ 2-bit experts) and
huihui-ai (abliterated trunk/embeddings). No endorsement implied.

Credits

If you use this model, please cite the quantization method — paper:
arXiv:2604.18556.

@article{gsq2026,
  title   = {GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling},
  author  = {Dadgarnia, Alireza and Tabesh, Soroush and Nikdan, Mahdi and Helcig, Michael and
             Kurti{\'c}, Eldar and Kleinegger, Max and Alistarh, Dan},
  journal = {arXiv preprint arXiv:2604.18556},
  year    = {2026},
  url     = {https://arxiv.org/abs/2604.18556}
}

README history 4 versions

The author's README evolved over time. Click a version to see its content at that point.

  1. 2026-11-16 Upload README.md with huggingface_hub 45c7307 6.3 KB
    Loading...
  2. 2026-11-16 Upload README.md with huggingface_hub 0d51a3d 5.1 KB
    Loading...
  3. 2026-11-16 Upload README.md with huggingface_hub 2012664 4.8 KB
    Loading...
  4. 2026-11-16 Upload folder using huggingface_hub 8dc89ff 4.5 KB
    Loading...
Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration