The model produces weird or garbled output

Short answer

Most likely: chat template mismatch. Fix in one step: open the model’s Edit dialog and switch the “Chat template” dropdown from Auto-detect to the correct family for your model. Second possibility: quantization too aggressive (Q2/Q3 sometimes fall apart) - download a Q4 or higher variant.

Detailed explanation

Symptoms of chat template mismatch:

• Visible special tokens in the output (<|im_end|>, <|eot_id|>, <start_of_turn>, etc.)
• Model hallucinates additional user/assistant turns
• Output starts fine then devolves into nonsense
• Model refuses to respond or generates a single character

Abliteration auto-detects the chat template on model add. For most modern models this works. When it doesn’t, the fix is direct - set the correct template manually.

Fix - manual template override:

1. Go to Models tab

2. Click ⋯ on the misbehaving model → Edit

3. Find the “Chat template” dropdown (currently on “Auto-detect”)

4. Switch to the correct family for your model:

   • Llama - for Llama-3, Llama-3.1, Llama-3.2 based models
   • Qwen - for Qwen 2.5, Qwen3 based models
   • ChatML - safe default for most modern models
   • Gemma - for Gemma 2, Gemma 3 based models
   • Mistral - for Mistral 7B, Mixtral based models
   • Phi - for Phi-3, Phi-3.5 based models
   • DeepSeek - for DeepSeek models
   • Or one of GLM, Kimi, Command-R, Yi, InternLM, Baichuan, OLMo, Falcon

5. Save. The change applies on next model load - close and reopen the chat, or send a new message.

If you don’t know your model’s family, check its Hugging Face page - the README almost always names the chat format. Or try ChatML first, then Llama or Qwen - one of them fits ~80% of modern models.

Symptoms of over-aggressive quantization:

• Model is grammatically fine but semantically nonsensical
• Repetitive phrasing, gets stuck in loops
• Forgets basic facts it should know
• Feels much dumber than the parameter count would suggest

Q2 and Q3 quantizations compress weights so hard that quality can drop off a cliff. Q4 is usually the safe floor.

Fix: download a Q4_K_M or higher variant. If your RAM can’t handle it, pick a smaller model at Q4 instead of the same model at Q2.

Symptoms of just-a-bad-model:

• Consistent low quality across topics with all templates
• Was already reported as low-quality by the community
• Fine-tune of a smaller base than claimed

Some GGUFs floating around Hugging Face are experimental, broken merges, or just bad.

Fix: check the model’s Hugging Face page for comments and downloads count. Well-tested models have thousands of downloads and comment threads. New/obscure ones may just not work.

Why it matters

“Weird output” has three causes with different fixes. Chat template mismatch is the most common and the easiest to fix - one dropdown change. Diagnosing which cause you have determines whether it’s a 10-second fix or a re-download.

In Abliteration specifically

The Chat template dropdown in the Edit dialog is always available. Auto-detect is the default; explicit family values override detection. This is a real fix, not a workaround.

Related

• Chat template auto-detection - how detection works
• Edit model dialog - where the dropdown lives
• Quantization - the second most common cause