license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.6-35B-A3B/blob/main/LICENSE
base_model:
- huawei-csl/Qwen3.6-35B-A3B-MACaQ-R-GGUF
- Qwen/Qwen3.6-35B-A3B
tags: - gguf
- abliterated
- uncensored
- macaq
- qwen3_5_moe
- llama.cpp
pipeline_tag: text-generation
Qwen3.6-35B-A3B-MACaQ-R-abliterated-GGUF
⚠️ Neither file has been behaviorally evaluated yet. Only structural and numerical checks have been done. Please compare KL divergence vs FP16 (e.g.
llama-perplexity --kl-divergence) against the original MACaQ-R and Unsloth UD-Q4_K_XL before relying on either.⚠️ 两个文件都还没做行为评测。 目前只做了结构和数值校验。使用前请先和原版 MACaQ-R、Unsloth UD-Q4_K_XL 比较一下相对 FP16 的 KL 散度。
English | 简体中文
Abliterated (refusal-direction-removed) variants of Huawei CSL's
MACaQ-R Q4_K_XL GGUF of
Qwen/Qwen3.6-35B-A3B. Both files have the same size, the same tensor types and the same metadata as the original MACaQ-R file. They run in stock llama.cpp (arch qwen35moe). The vision projector and the MTP draft head are provided as separate files (see below).
| Variant | File | Size | SHA-256 |
|---|---|---|---|
| A – constrained re-encode | Qwen3.6-35B-A3B-MACaQ-R-abliterated-Q4_K_XL.gguf |
22,360,455,968 B | b4d2f32848f945de77828efcd3376ce4f773189acba68f01e4a25d1e92814b42 |
| B – huihui down_exps transplant | Qwen3.6-35B-A3B-MACaQ-R-abliterated-huihuiDown-Q4_K_XL.gguf |
22,360,455,968 B | 5bf4308b92b26cc39945b5d88aa143a12ff793e0e7d39999122eb9723ac1daeb |
Extra files: vision and MTP
| File | Size | SHA-256 | Source |
|---|---|---|---|
mmproj-Qwen3.6-35B-A3B-Q8_0.gguf |
614,194,304 B | 904cbf8c8e876220066ab3bf676c7efa40f3da372276fdaf8b01d2fb2a37a51d |
ggml-org/Qwen3.6-35B-A3B-GGUF, unmodified (Apache-2.0) |
mmproj-Qwen3.6-35B-A3B-BF16.gguf |
902,822,144 B | 360c746ddf38fb67c06f83a0a88742492aa231e59e335cd980e6302f59ea6089 |
ggml-org, unmodified |
mtp-Qwen3.6-35B-A3B-MACaQ-R-abliterated-Q8_0.gguf |
1,990,649,920 B | 81ac72d26037c2c0c0ed78c4675afed2004ac1b41deeb8ce9b9878ebcd53591c |
ggml-org's MTP draft head, with huihui's abliterated tensors swapped in |
- Vision (mmproj). huihui's abliteration did not touch the vision tower: all 333
visual.*tensors in the huihui BF16 checkpoint are byte-identical to Qwen's. So the stock projector is correct for both variants (qwen3vl_merger, projection_dim 2048). - MTP. Upstream llama.cpp loads the Qwen3.6 MTP head as a separate "MTP-only" sidecar GGUF (
blk.40.*plus embeddings and output head,nextn_predict_layers = 1), used with--spec-type draft-mtp. huihui's MTP GGUF shows that huihui also abliterated the MTP block (blk.40.attn_output,ffn_down_expsandffn_down_shexpdiffer from stock). I copied those 3 Q8_0 tensors byte for byte from huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated-MTP-GGUF into ggml-org's MTP file. Checks: the same rank-1 edit with the same r (α≈1.004), and |rᵀW| left below 1%. The other 20 tensors are byte-identical to ggml-org's file. - Tested on CPU with upstream llama.cpp (commit
4e5116f, 2026-11-15), using--no-repack -lm mmap:- Both A and B answered the test image correctly ("A red square and a blue circle.").
- The MTP draft loaded and was used: draft acceptance was 47/47 on a counting prompt and 37/69 (54%) on free text.
- mmproj and MTP also worked together in
llama-server. - On that 8-core CPU, MTP gave no speed-up (5.3 vs 6.6 tok/s), because verifying drafts is expensive on CPU. Any speed-up on GPU is not measured.
Usage (upstream llama.cpp)
# text (variant A or B)
llama-cli -m Qwen3.6-35B-A3B-MACaQ-R-abliterated-huihuiDown-Q4_K_XL.gguf
# vision
llama-mtmd-cli -m Qwen3.6-35B-A3B-MACaQ-R-abliterated-huihuiDown-Q4_K_XL.gguf \
--mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf --image photo.jpg -p "Describe this image."
# server with vision + MTP speculative decoding
llama-server -m Qwen3.6-35B-A3B-MACaQ-R-abliterated-huihuiDown-Q4_K_XL.gguf \
--mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
-md mtp-Qwen3.6-35B-A3B-MACaQ-R-abliterated-Q8_0.gguf --spec-type draft-mtp --spec-draft-n-max 3
Tip: on low-RAM machines, add --no-repack -lm mmap so the weights are memory-mapped instead of copied into RAM.
Shared method
The abliteration source is huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated, via the GGUFs from mradermacher/Huihui-Qwen3.6-35B-A3B-abliterated-GGUF. The tensor-transplant idea comes from SC117.
- What huihui changed. The edit is exactly a rank-1 projection W ← W − r rᵀW, with one direction r shared by all 40 layers. It touches only the 120 tensors that write to the residual stream:
attn_output×10,ssm_out×30,ffn_down_shexp×40,ffn_down_exps×40. Embeddings,output, the router and the gate/up experts are untouched. - 80 Q8_0 tensors (both variants): copied byte for byte from the mradermacher Q8_0. MACaQ-R only refines Q4_K/Q5_K/Q6_K codes, and its Q8_0 tensors are identical to stock llama.cpp Q8_0, so nothing MACaQ-R did is lost here.
- 613 other tensors, header and metadata: byte-identical to MACaQ-R. The gate/up experts keep MACaQ-R's refinement in full (about 2/3 of MACaQ-R's refined weights).
The variants differ only in the 40 ffn_down_exps tensors
A – constrained re-encode.
- For each of the 10,240 experts, I applied W − r rᵀW to MACaQ-R's own dequantized weights.
- I kept every MACaQ-R scale and min unchanged and greedily changed integer codes by ±1 (unweighted squared error) until rᵀW ≈ 0.
- Result: about 8.5% of codes changed (Q5_K 7.5%, Q6_K 17.4%), refusal component left 0.23% on average (max 0.93%).
- Honest caveat: MACaQ-R optimizes an activation-Hessian-weighted error (see
llamacpp/refinement.pyin the MACaQ repo). The ideal edit itself keeps MACaQ-R's error structure, but the discrete flips add rounding noise. My estimate is that A'sffn_down_expsend up roughly 1.2–1.35× Unsloth UD-Q4_K_XL's error on MACaQ-R's own objective, which is worse than both MACaQ-R and Unsloth for those tensors. This rests on a diagonal-imatrix proxy plus an assumed MACaQ-R gain of 15–30%; it was not measured.
B – huihui down_exps transplant.
- The 40
ffn_down_expstensors are copied byte for byte from mradermacher's static quants of the huihui model. The types match MACaQ-R exactly: Q5_K fromQ5_K_Sfor 36 layers, and Q6_K fromQ6_Kfor layers 1, 34, 38 and 39. - No re-encoding, so the file size and offsets are unchanged.
- MACaQ-R's refinement is not used in these tensors (they are plain static quants), but no flip noise is added either.
- I checked the alternatives on 18 sampled experts against the ideal abliterated BF16 weights:
- Re-encoding huihui's Q6_K onto MACaQ's Q5_K grid gave about 1.29–1.34× Unsloth error, which is worse than a direct Q5_K copy.
- Keeping Q6_K would have lower error but makes the file about 1.3 GB larger.
- Measured error of B's
ffn_down_expsvs the ideal abliterated weights (diagonal-imatrix proxy, relative to Unsloth's error vs the original weights): 1.02–1.09× (Q5_K) and 1.06–1.11× (Q6_K). On the same proxy, A comes out at 1.58–1.70× (Q5_K) and 1.90–2.50× (Q6_K), and the original MACaQ-R at 1.10–1.22×.- Caution: this proxy penalizes MACaQ-R's correlated error and ignores its benefit, so it cannot rank MACaQ-R against Unsloth. It does show that A carries clearly more noise than B.
- B's refusal component left (|rᵀW| / |rᵀW_Unsloth|, all 10,240 experts): Q5_K layers mean 3.6% (max 4.9%), Q6_K layers mean 2.0% (max 2.6%). These are the same levels as huihui's own quants.
Expectation (unverified): B should have lower error than A in ffn_down_exps, while A removes the refusal direction more completely (0.2% vs 2–4% left). Overall, both should sit between MACaQ-R and plain Unsloth quality. Please measure KL vs FP16 and pick whichever is better for you.
Verification (structural)
- A:
- Metadata (55 keys), tensor table (733 tensors) and header are identical to MACaQ-R.
- Per-tensor blake2b: 120 changed, 613 identical; 80/80 Q8_0 targets match the source.
- Codes changed: Q5_K mean 7.54% (max 8.78% in L14), Q6_K mean 17.39% (max 17.70% in L39).
- B:
- Loads with the
ggufPython reader. Metadata, tensor table and header are identical to A and MACaQ-R. - The 693 non-target tensors are byte-identical to A.
- All 40
ffn_down_expsare byte-identical to the mradermacher source; an independent second download matched 40/40.
- Loads with the
⚠️ Safety notice
These models have had refusals removed and have no guardrails. They may produce harmful, false, offensive or illegal content. They are for research and private local use only. Do not deploy them publicly or give them to minors. You are fully responsible for how you use them and must comply with your local laws.
Credits
Qwen team (Qwen3.6-35B-A3B) · ggml-org (mmproj and MTP GGUFs) · Huawei CSL (MACaQ, MACaQ-R GGUF) · huihui-ai (abliterated weights) · mradermacher (GGUF quants) · SC117 (tensor-transplant idea). License: Apache-2.0.
简体中文
这里是华为 CSL MACaQ-R Q4_K_XL GGUF 的两个消融版(去除拒答方向)。两个文件的大小、张量类型和元数据都和原版一样,普通 llama.cpp 就能跑;视觉编码器和 MTP 草稿头是单独的附加文件。
视觉与 MTP 附加文件:
mmproj-Qwen3.6-35B-A3B-Q8_0.gguf和mmproj-Qwen3.6-35B-A3B-BF16.gguf:视觉编码器,来自 ggml-org,未做修改。huihui 的消融没有动视觉部分(333 个视觉张量和 Qwen 原版逐字节一致),所以 A 和 B 都能直接用。mtp-Qwen3.6-35B-A3B-MACaQ-R-abliterated-Q8_0.gguf:MTP 推测解码草稿头。在 ggml-org 的 MTP 文件基础上,换入了 huihui 消融过的 3 个张量(huihui 连 MTP 层也一起消融了)。- 已在 CPU 上用最新 llama.cpp 实测:
- A 和 B 都能正确识别测试图片。
- MTP 草稿头能正常加载并被采用(接受率:数数任务 47/47,普通文本 54%)。
- 在这台 8 核 CPU 上 MTP 没有提速;在 GPU 上能否提速还没测。
用法:
llama-mtmd-cli -m <A或B>.gguf --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf --image 图片.jpg -p "描述这张图片"
llama-server -m <A或B>.gguf --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf -md mtp-Qwen3.6-35B-A3B-MACaQ-R-abliterated-Q8_0.gguf --spec-type draft-mtp
两个版本的共同点:
- huihui 的消融是秩 1 投影 W ← W − r rᵀW,40 层共用同一个方向 r,只改了 120 个张量。
- 其中 80 个 Q8_0 张量直接从 mradermacher 的 Q8_0 逐字节拷贝。MACaQ-R 本来就没优化这些张量,所以没有损失。
- gate/up 专家等 613 个张量和原版完全一样,MACaQ-R 的优化全部保留。
两个版本只在 40 个专家 down 张量上不同:
- A(约束重编码): 保留 MACaQ-R 的缩放系数,翻动约 8.5% 的编码,让拒答方向几乎清零(平均只剩 0.23%)。不过 MACaQ-R 优化的是按激活加权的误差,我们的翻动会带来额外的取整噪声。估计这些层的误差约为 Unsloth 的 1.2~1.35 倍,这个数字没有实测过。
- B(huihui down 张量移植): 直接拷贝 mradermacher 静态量化里类型完全相同的张量(36 层 Q5_K、4 层 Q6_K),不需要重编码。这些层不再有 MACaQ 的优化,但也没有额外噪声。实测误差约为 Unsloth 的 1.02~1.11 倍,拒答成分剩 2~4%,和 huihui 自己的量化版一样。
预期(未验证): B 在专家 down 层的误差应该比 A 小,A 去拒答更彻底。两者整体都应该介于 MACaQ-R 和普通 Unsloth 之间。请自己测一下相对 FP16 的 KL 散度,再选更好的那个。
⚠️ 安全提示: 本模型没有安全护栏,可能生成有害、错误或违法的内容。仅供研究和个人本地使用,请勿公开部署,也不要提供给未成年人。使用者须自行承担全部责任。
致谢:Qwen 团队、ggml-org(视觉编码器和 MTP 文件)、华为 CSL(MACaQ)、huihui-ai、mradermacher、SC117。许可证:Apache-2.0。