license: apache-2.0
base_model:
- Qwen/Qwen3.6-35B-A3B
- HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
language: - en
tags: - gguf
- qwen3.6
- moe
- llama.cpp
- quantization
- vocabulary-pruned
- tokenizer
- uncensored
- amd
- vulkan
- rdna1
- local-llm
Monoglot 35B-A3B: Qwen3.6-35B-A3B with its vocabulary cut to what English actually uses
One language, on purpose. Third entry in the Monoglot series, after the
27B and the
9B.
This is the entry I expected least and measured the most: only 1.6% of the file came off, and it bought
+26% decode and +62% prompt processing. On a Mixture-of-Experts model that spills most of itself into
system memory, the output head is read in full every single token, so halving the head is not a
file-size trick.
118,300 of 248,320 vocabulary rows removed. Every surviving weight row is bit-identical to the source.
Credit, in order of who did what
- The idea and the first public build: fakezeta, whose
Condensed-P2release is what
showed this was worth doing. - The tool:
bsaleh03/ASCII-Condensed-prune-tools:
the policies, keep-set construction and in-quantised-space row gather are his work. That repository has
no licence, so none of its code is redistributed here; it is linked as the credit it is. - @TeksEdge for the post that surfaced it.
- The model: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
on base Qwen/Qwen3.6-35B-A3B. The uncensored tuning is
theirs and a vocab prune neither adds to nor subtracts from it.
Mine is the application to this file, the measurements, and the parts that did not work.
The model, for context
35B total, about 3B active, 256 experts with 8 routed per token, hidden 2048. On a 7.98 GiB card it cannot
fit: it runs with about 7.6 GiB in VRAM and 12.9 GiB spilled to GTT, and it is usable only because each
token activates 8 of 256 experts.
That is the setup that makes this prune pay so much.
| vocab rows | vocab bytes | share of file | |
|---|---|---|---|
| Source | 248,320 | 703 MB | 3.3% |
| This file | 130,020 | 368 MB | 1.8% |
| Tensor | Type | Before | After |
|---|---|---|---|
output.weight |
Q6_K | 417 MB | 218 MB |
token_embd.weight |
Q4_K | 286 MB | 150 MB |
| File | Q4_K_M | 21,155,768,832 B | 20,825,407,520 B |
330 MB smaller, 1.6%. Policy is P1G: ASCII plus math and typography symbols, Greek, box-drawing and
status glyphs, units and currency.
What it bought, measured
llama.cpp 3ba1ce0 Vulkan, RX 5700 XT (8 GB, RDNA1), -ngl 999 -ctk q4_0 -ctv q4_0 -fa on -t 6 --cpu-mask 0x3f, GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM=1, measured at depth 4096. Three reps, card empty
before each run.
| llama-bench | source | this file | |
|---|---|---|---|
| pp512 @ 4096 | 91.95 ± 0.36 | 149.38 ± 3.02 | +62% |
| tg128 @ 4096 | 10.16 ± 0.01 | 12.76 ± 0.01 | +26% |
| GTT in use while running | 12.94 GiB | 12.09 GiB |
Why 1.6% of the file moves the needle: decode here is not a compute problem, it is a transfer problem.
The output head has to be read in full for every generated token no matter how sparse the experts are, and
on this card most of the model sits in GTT over PCIe 3.0 at roughly 13 GB/s. Halving a 417 MB head is
199 MB less per token, which at 13 GB/s is about 15 ms of the ~98 ms each token was taking. Measured gain
was 26%, a little better than that arithmetic, and the prefill gain (62%) is larger still because prefill
was thrashing GTT harder.
So the three entries in this series are three different mechanisms, and none of them was the one the
original post described:
| what the prune actually bought | |
|---|---|
| 27B | crossed back under the VRAM wall: prefill 6.2x, decode 2.4x |
| 9B | less per-token bandwidth, no wall to cross: decode +6.1%, prefill flat |
| 35B-A3B | halved the one tensor read every token on an already-spilling MoE: decode +26%, prefill +62% |
What it costs
| Text | Source | This file | Ratio |
|---|---|---|---|
| English prose | 30 | 30 | 1.00x |
| Python | 41 | 41 | 1.00x |
Math with ∀ ∑ π ≥ ✓ |
27 | 27 | 1.00x |
| French, accented | 18 | 31 | 1.72x |
| Japanese | 15 | 40 | 2.67x |
| Chinese | 15 | 50 | 3.33x |
| Russian | 17 | 93 | 5.47x |
Same table as every other entry, because the policy is the same. If you need multilingual output, use the
source file.
Correctness checks that did pass
verify_prune.py --policy P1G: all 731 non-vocabulary tensors byte-identical, gathered rows match their
source rows, keep-set exactly the policy, every metadata key preserved.- Greedy 64-token continuations byte-identical to the source for an English prompt and a code prompt.
- A
toolsrequest on both files returnsfinish_reason: tool_calls,read_file, and{"path":"/etc/hostname"}.
What I did not measure
- No instruction-following benchmark on this file. Correctness rests on the checks above.
- No perplexity or KL, because neither is comparable across a vocabulary change.
- A different build and depth from my own production profile for this model. The numbers above are one
binary at depth 4096 with the placement fix applied correctly; my production launch of the same family
uses another build at 32K. The A/B is like-for-like within this table, and the absolute rates are not
transferable to a different binary. - No
mmproj, text only. - Refusal behaviour not tested.
Using it
llama-server -m Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_M-Monoglot-P1G-130K.gguf \
-c 32768 -ngl 999 -ctk q4_0 -ctv q4_0 -fa on
It wants the same treatment as the source: all layers offloaded, KV quantised, and the placement flag spelled
exactly GGML_VK_DISABLE_HOST_VISIBLE_VIDMEM. A near-miss spelling is accepted by the shell, sets a variable
nothing reads, and silently puts you back in the slow regime.
Files and verification
| File | Bytes | SHA-256 |
|---|---|---|
Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive-Q4_K_M-Monoglot-P1G-130K.gguf |
20,825,407,520 | b9876abbe198501fd08ed5c775e2762590ad4e5047a419aeac2a91ebe41267ae |
Licence
Apache-2.0, following the base model. Weights derive from Qwen3.6-35B-A3B and HauhauCS's uncensored tuning.
The pruning tool is linked, not included, because it carries no licence.