license: apache-2.0
pipeline_tag: text-generation
base_model: Qwen/Qwen3.8-27B
tags:
- llamafile
- gguf
- qwen3.8
I am not the original creator of llamafile, all credit of llamafile goes to Jartine:
jartine's LLM work is generously supported by a grant from mozilla
OBLITERATUS GGUF - llamafile
Run LLMs locally with a single file
Download a file and run it. Llamafile combines llama.cpp with Cosmopolitan Libc in a single executable.
How to use
Open a Linux terminal.
Grant execute permission.
chmod +x Qwen3.8-27B-OBLITERATED-Q4_K_M.llamafile
- Run a text prompt with thinking disabled.
prompt='<|im_start|>user
What is the capital of France? Answer with only the city name.<|im_end|>
<|im_start|>assistant
<think>
</think>
'
/bin/sh Qwen3.8-27B-OBLITERATED-Q4_K_M.llamafile \
--cli --no-display-prompt --no-conversation --log-file /dev/null \
-ngl 999 -c 512 -n 64 --temp 0 -p "$prompt"
- Press Control-C to stop generation.
Settings for OBLITERATUS GGUF Llamafiles
- Model creator: OBLITERATUS
- Original model: Qwen/Qwen3.8-27B
- Modified model: OBLITERATUS/Qwen3.8-27B-OBLITERATED
- Built by Pliny the Prompter.
- Quantized GGUF files used: OBLITERATUS/Qwen3.8-27B-OBLITERATED
- Commit hash
a58c3b53b3ce71551eafde2ed5ec8df48e0f4ff8 - Commit message: V3: tokenizer_config.json with correct chat template
- Commit hash
- Llamafile version: 0.10.5, mozilla-ai/llamafile
- Commit hash
486e6c5f9356eae50b851b07517bfae1f2420193 - Commit message: docs(support): document the Vulkan GPU backend (#1029) (#1033)
- Commit hash
- Quantizations: Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0.
- Validation: Linux text generation; details in
validation/. - Vision: text GGUF; upstream projector files remain separate.
- MTP: source tensors retained; validation uses standard decoding.
.argscontents:
-m
Qwen3.8-27B-OBLITERATED-Q4_K_M.gguf
...
(Following is original model card for OBLITERATUS GGUF)
Excerpt from the original GGUF source card: OBLITERATUS/Qwen3.8-27B-OBLITERATED.
🧨 How It Works — V1 → V2 → V3
Abliteration removes refusal behavior by identifying and projecting out "refusal directions" from the model's weight space. Each version refined the approach:
V1: Single Surgery
One aggressive SVD pass with 5 directions. Removed hard refusals completely but cost -6pp MMLU — the model got noticeably dumber.
V2: Complementary Blending
The breakthrough: run TWO different surgeries that fail in different ways, then blend their weights. SVD captures refusal greedily (damages capability). LEACE minimizes mutual information (preserves capability but weaker refusal removal). Blending at 60/40 cancels each method's weaknesses — a novel technique we call complementary abliteration blending.
Result: -0.3pp MMLU (near-stock) but still deflected on some simple queries with safety lectures instead of hard refusals.
V3: Iterative Refinement + Targeted Surgery
Two key insights:
- Iterative stacking — refine the champion model, never start from stock. Each surgery round builds on previous rounds' gains.
- Targeted corpus — use a focused corpus for specific deflection categories to find their unique refusal directions without diluting the signal.
V3 applies gentle iterative refinement on V2, then a targeted surgery pass with a focused corpus, then blends the results. This eliminated not just hard refusals ("I cannot") but also soft deflections (safety lectures that give zero substance).
Result: -2.1pp MMLU — a modest cost for genuine liberation across all categories.