license: apache-2.0
base_model: abacusai/Smaug-Political-Uncensored
base_model_relation: quantized
pipeline_tag: image-text-to-text
language:
- en
- zh
tags: - gguf
- llama.cpp
- mtp
- agentic
- smaug
- abacusai
1. Introduction
GGUF builds of Smaug-Political-Uncensored for llama.cpp, Ollama and LM Studio: seven sizes from IQ3_M to Q8_0, each as a plain file and as a file that keeps the model's multi-token-prediction (MTP) draft head for faster generation, plus the vision projector for image input. Every file was loaded and tested with thinking on, thinking off and an image, and the Q4_K_M file was rerun through the main card's full refusal and political evaluation and through AutomationBench (§4). What the model does and how it was trained is on the main card.
Built with llama.cpp 71a6a11 from the BF16 weights, with an importance matrix from bartowski's calibration_datav3 text (imatrix.gguf, included).
2. Which file to pick
| Quant | Plain file | With MTP head | Mean KLD | 99% KLD | Same top token | Pick it when you have |
|---|---|---|---|---|---|---|
IQ3_M | 11.7 GiB | 11.9 GiB | 0.0577 | 0.557 | 90.8% | Smallest file. 16 GB of GPU memory with a 32K context; a 12 GB card needs part of the model on the CPU. |
Q3_K_M | 12.4 GiB | 12.6 GiB | 0.0471 | 0.471 | 91.0% | 16 GB of GPU memory with a context up to about 24K. Closer to full precision than IQ3_M. |
IQ4_XS | 14.0 GiB | 14.3 GiB | 0.0162 | 0.172 | 94.9% | 16 GB of GPU memory with a short context. |
Q4_K_M | 15.4 GiB | 15.7 GiB | 0.0124 | 0.130 | 95.4% | The default (Ollama picks it). 24 GB of GPU memory. |
Q5_K_M | 17.9 GiB | 18.2 GiB | 0.0045 | 0.045 | 97.2% | 24 GB with room for a longer context. |
Q6_K | 20.6 GiB | 20.9 GiB | 0.0022 | 0.020 | 98.1% | 32 GB, or a 24 GB card with a short context. |
Q8_0 | 26.6 GiB | 27.1 GiB | 0.0008 | 0.008 | 98.8% | Close to full precision. 32 GB and up, or a 48 GB Mac. |
llama-perplexity --kl-divergence; lower is closer. "Same top token" is how often the quant's most likely next token equals BF16's. The MTP files quantize the language model exactly like the plain files (every shared tensor is byte-identical) and add the 15 tensors of the draft head. Memory needed is about the file size plus 64 KiB per token of context (16 full-attention layers) plus 0.15 GiB per parallel sequence for the linear-attention state, plus 0.9 GiB for the vision projector when images are used.3. Quick start
llama.cpp (downloads the vision projector too):
llama-server -hf abacusai/Smaug-Political-Uncensored-GGUF:Q4_K_M --jinja -c 32768 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0
The model thinks by default (reasoning effort xhigh). To turn thinking off, add --chat-template-kwargs '{"enable_thinking": false}', or send "chat_template_kwargs": {"enable_thinking": false} with a request.
Faster generation with the MTP head (llama.cpp): use a file from MTP/ and turn on MTP drafting.
llama-server --hf-repo abacusai/Smaug-Political-Uncensored-GGUF --hf-file MTP/Smaug-Political-Uncensored-MTP-Q4_K_M.gguf \
--mmproj-url https://huggingface.co/abacusai/Smaug-Political-Uncensored-GGUF/resolve/main/mmproj-Smaug-Political-Uncensored-F16.gguf \
--jinja -c 32768 --spec-type draft-mtp --temp 1.0 --top-p 0.95 --top-k 20 --min-p 0
Ollama:
ollama run hf.co/abacusai/Smaug-Political-Uncensored-GGUF:Q4_K_M
LM Studio: search for abacusai/Smaug-Political-Uncensored-GGUF and pick a file from the table above.
4. Checks on the quantized files
Refusals and political answers, Q4_K_M vs BF16
Quantising to Q4_K_M did not shift refusals. Scored prompt by prompt against BF16 (paired exact tests): 1,451 vs 1,433 of 3,355 refusal-battery prompts refused with thinking on (p 0.36) and 1,466 vs 1,462 with thinking off (p 0.86); 1,992 vs 2,002 of 4,330 political prompts answered factually with thinking on (p 0.72) and 1,439 vs 1,440 of 4,339 with thinking off (p 1.0). Requests for chemical, biological, radiological or nuclear weapons (40) and for child sexual abuse material (23) are refused every time in both modes. Of 54 per-set tests (27 prompt sets, two modes), one moved significantly, in the factual direction: China-censored questions with thinking on, 202 vs 183 of 267 (p 0.025), about what chance alone gives across that many tests.
Thinking on (the default).
| Q4_K_M llama.cpp | BF16 vLLM | Qwen3.8-27B (base, BF16) |
|
|---|---|---|---|
| Answers about China's politics, history and leaders | |||
| China-censored questions answered factually deccp held-out, en + zh_cn + zh_tw |
75.7 202/267 |
68.5 183/267 |
44.9 120/267 |
| Held-out political questions answered factually 224 questions x 3 scripts (en, zh_cn, zh_tw) |
89.9 602/670 |
89.7 599/668 |
68.0 457/672 |
| Pan & Xu political questions answered factually en + zh_cn |
78.3 227/290 |
80.0 232/290 |
56.2 163/290 |
| Held-out political answers framed as the PRC position same 672 questions; lower is better |
5.7 38/670 |
5.8 39/668 |
19.2 129/672 |
| promptfoo CCP-sensitive prompts answered factually held-out 966 prompts x 3 scripts |
28.4 824/2897 |
29.7 859/2896 |
22.3 646/2896 |
| Refusals of requests that should be answered (lower is better) | |||
| Satire of China's leaders refused 250 held-out requests, angles never trained |
26.4 66/250 |
28.4 71/250 |
67.2 168/250 |
| Satire of other governments refused 36 held-out requests |
8.3 3/36 |
5.6 2/36 |
16.7 6/36 |
| Same request about China refused 30 templates, each also asked about the US |
53.3 16/30 |
43.3 13/30 |
83.3 25/30 |
| Same request about the US refused the US half of the same 30 templates |
0.0 0/29 |
3.3 1/30 |
0.0 0/29 |
| Benign but risky-looking prompts refused OR-Bench-Hard-1K |
26.3 347/1319 |
24.9 328/1318 |
45.2 596/1318 |
| Safe prompts refused XSTest safe split |
5.6 14/250 |
6.0 15/249 |
6.0 15/250 |
| Refusals of harmful requests (higher is better) | |||
| Chemical, biological, radiological, nuclear weapons held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
100.0 40/40 |
100.0 40/40 |
100.0 40/40 |
| Child sexual abuse material held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
100.0 23/23 |
100.0 23/23 |
100.0 23/23 |
| All 15 harmful categories pooled held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
94.1 777/826 |
93.5 772/826 |
90.2 745/826 |
| HarmBench standard behaviors |
96.9 154/159 |
95.6 152/159 |
95.6 152/159 |
| JailbreakBench harmful behaviors, AdvBench rows dropped |
97.6 80/82 |
97.6 80/82 |
95.1 78/82 |
Thinking off.
| Q4_K_M llama.cpp | BF16 vLLM | Qwen3.8-27B (base, BF16) |
|
|---|---|---|---|
| Answers about China's politics, history and leaders | |||
| China-censored questions answered factually deccp held-out, en + zh_cn + zh_tw |
29.2 78/267 |
25.5 68/267 |
9.4 25/267 |
| Held-out political questions answered factually 224 questions x 3 scripts (en, zh_cn, zh_tw) |
79.3 533/672 |
81.2 546/672 |
42.4 285/672 |
| Pan & Xu political questions answered factually en + zh_cn |
64.5 187/290 |
65.2 189/290 |
29.3 85/290 |
| Held-out political answers framed as the PRC position same 672 questions; lower is better |
16.1 108/672 |
15.8 106/672 |
42.3 284/672 |
| promptfoo CCP-sensitive prompts answered factually held-out 966 prompts x 3 scripts |
17.9 520/2898 |
18.1 525/2898 |
7.7 223/2898 |
| Refusals of requests that should be answered (lower is better) | |||
| Satire of China's leaders refused 250 held-out requests, angles never trained |
24.8 62/250 |
30.0 75/250 |
76.8 192/250 |
| Satire of other governments refused 36 held-out requests |
5.6 2/36 |
5.6 2/36 |
55.6 20/36 |
| Same request about China refused 30 templates, each also asked about the US |
13.3 4/30 |
23.3 7/30 |
93.3 28/30 |
| Same request about the US refused the US half of the same 30 templates |
3.3 1/30 |
0.0 0/30 |
23.3 7/30 |
| Benign but risky-looking prompts refused OR-Bench-Hard-1K |
26.7 352/1319 |
25.4 335/1319 |
75.9 1001/1319 |
| Safe prompts refused XSTest safe split |
5.6 14/250 |
6.4 16/250 |
10.0 25/250 |
| Refusals of harmful requests (higher is better) | |||
| Chemical, biological, radiological, nuclear weapons held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
100.0 40/40 |
100.0 40/40 |
100.0 40/40 |
| Child sexual abuse material held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
100.0 23/23 |
100.0 23/23 |
100.0 23/23 |
| All 15 harmful categories pooled held-out CatQA, AIR-Bench, HarmBench, JBB prompts |
95.3 787/826 |
95.4 788/826 |
96.1 794/826 |
| HarmBench standard behaviors |
98.1 156/159 |
97.5 155/159 |
98.7 157/159 |
| JailbreakBench harmful behaviors, AdvBench rows dropped |
95.1 78/82 |
96.3 79/82 |
97.6 80/82 |
Agentic tool use: AutomationBench
| Q4_K_M llama.cpp | BF16 vLLM | Qwen3.8-27B (base, BF16, vLLM) | |
|---|---|---|---|
| Tasks fully completed | 36.2 (217/600) | 38.6 (463/1200, 2 runs) | 38.3 (230/600) |
| Mean partial credit | 0.768 | 0.772 | 0.773 |
MTP draft head
The MTP head is inherited from the base model, and it still predicts this model's tokens well. With 3 draft tokens in vLLM 0.32.0, this model accepts 62.2% of drafted tokens (mean accepted length 2.86 tokens per step) against 58.7% (2.76) for the base model, on the same 150 prompts (IFEval, GSM8K, HumanEval) with thinking on. In llama.cpp's smoke test of the MTP Q4_K_M file, 117 of 162 drafted tokens were accepted.
Chat template
llama.cpp renders this repo's embedded chat template exactly as the Hugging Face tokenizer does (checked for a thinking-on turn, a thinking-off turn and a tool-call round trip), so tool calling and the thinking switch behave as in the BF16 release.
5. License
Apache 2.0, inherited from Qwen3.8-27B. See LICENSE and NOTICE.
6. Citation
@misc{abacusai2026smaugPoliticalUncensored,
title = {Smaug-Political-Uncensored},
author = {Abacus.AI},
year = {2026},
note = {Fine-tune of Qwen/Qwen3.8-27B that answers China-political questions factually and refuses fewer benign requests},
url = {https://huggingface.co/abacusai/Smaug-Political-Uncensored}
}
7. Contact Us
If you have any questions, please reach out at Abacus.AI.