license: apache-2.0
pipeline_tag: text-generation
base_model: Qwen/Qwen3.8-27B
tags:

• llamafile
• gguf
• qwen3.8

I am not the original creator of llamafile, all credit of llamafile goes to Jartine:

Chat & support: jartine's Discord server

jartine's LLM work is generously supported by a grant from mozilla

---

Huihui Abliterated GGUF - llamafile

Run LLMs locally with a single file

Download a file and run it. Llamafile combines llama.cpp with Cosmopolitan Libc in a single executable.

How to use

1. Download the llamafile.

2. Open a Linux terminal.

3. Grant execute permission.

chmod +x Huihui-Qwen3.8-27B-abliterated-Q4_K_M.llamafile

4. Run a text prompt with thinking disabled.

prompt='<|im_start|>user
What is the capital of France? Answer with only the city name.<|im_end|>
<|im_start|>assistant
<think>

</think>

'
/bin/sh Huihui-Qwen3.8-27B-abliterated-Q4_K_M.llamafile \
  --cli --no-display-prompt --no-conversation --log-file /dev/null \
  -ngl 999 -c 512 -n 64 --temp 0 -p "$prompt"

5. Press Control-C to stop generation.

Settings for Huihui Abliterated GGUF Llamafiles

• Model creator: huihui-ai
• Original model: Qwen/Qwen3.8-27B
• Quantized GGUF files used: huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF
  • Commit hash 8f1b52408a2f6e317535190c9386f776cacf0079
  • Commit message: Add files using upload-large-folder tool
• Llamafile version: 0.10.5, mozilla-ai/llamafile
  • Commit hash 486e6c5f9356eae50b851b07517bfae1f2420193
  • Commit message: docs(support): document the Vulkan GPU backend (#1029) (#1033)
• Quantizations: Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0.
• Validation: Linux text generation; details in validation/.
• Vision: text GGUF; upstream projector files remain separate.
• MTP: source tensors retained; validation uses standard decoding.
• .args contents:

-m
Huihui-Qwen3.8-27B-abliterated-Q4_K.gguf
...

(Following is original model card for Huihui Abliterated GGUF)

Excerpt from the original GGUF source card: huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF.

This is an uncensored version of Qwen/Qwen3.8-27B created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.

[…]

Note

The first 15 layers were retained without ablation. MTP and visual has not been modified.

We have already converted the weights (token_embd,output,ffn_down,ssm_out,attn_output) that need to be ablated in the versions below Q8_0 from Q2_K, Q3_K, Q4_K, Q5_K, and Q6_K to Q8_0 to improve response quality, and changed the filename to K_L.

In the Q8_0 quantized version, we changed the Q8_0 weights (token_embd,output,ffn_down,ssm_out,attn_output) targeted for ablation to BF16 and renamed the file to Q8_0_L.

This is not a standard quantization, so you might find that Q2_K_L is larger than Q3_K and Q4_K.