Chat templates - auto-detection and manual override

Short answer

Every model expects prompts wrapped in a specific format (Llama-style, ChatML, Gemma-style, etc.). Getting this wrong = garbled or refused output. Abliteration auto-detects the template on model add, but also gives you a manual override - a 16-option dropdown in the Edit-model dialog if auto-detection is wrong.

Detailed explanation

When you type “Hello”, the model doesn’t actually see “Hello”. It sees a formatted string like:

<|start_header_id|>user<|end_header_id|>

Hello<|eot_id|><|start_header_id|>assistant<|end_header_id|>

That’s the Llama 3 template. Qwen uses:

<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant

Gemma, Phi, DeepSeek, Mistral, Command-R, Yi - each has its own format. The special tokens are what the model was trained to recognise as “user speaking” vs “assistant speaking”. Feed a Qwen model a Llama-style prompt and it doesn’t know when the user’s message ended - output becomes chaotic.

Detection order in Abliteration (highest priority first):

1. Manual override - if you set a template explicitly in the Edit-model dialog, that wins.
2. Embedded template marker tokens - Abliteration looks at the GGUF’s tokenizer.chat_template field for marker tokens (<start_of_turn> → Gemma, <|im_start|> → ChatML/Qwen, etc.) and picks a matching built-in.
3. Filename heuristics - the filename’s family hint (e.g. llama-3, qwen2.5) is used if no markers matched.
4. Fallback - ChatML/Qwen format as default.

Note: Abliteration does NOT render the Jinja2 template from the GGUF directly. It pattern-matches the template’s structure to one of 12 built-in builders. This is more reliable than Jinja rendering for the common case.

Built-in template dropdown (16 options in Edit dialog):

Auto-detect · ChatML · Qwen · Llama · Mistral · DeepSeek · Gemma · Phi · GLM · Kimi · Command-R · Yi · InternLM · Baichuan · OLMo · Falcon

Yi and InternLM appear in the list but internally reuse the ChatML/Qwen builders (they use the same format). The 12 distinct builders cover the major families.

Where to set it manually:

Models tab → click ⋯ on a model → Edit → Chat template dropdown. Change from “Auto-detect” to a specific family. Saves immediately. Applies on next model load.

Why it matters

The #1 cause of “why is my model outputting garbage?” is a chat template mismatch. Auto-detection handles the common case; the manual override is the fix for the exceptions. If a model produces obvious template artifacts (visible special tokens in output, hallucinated user turns), switching the template dropdown to the correct family usually resolves it in one step.

In Abliteration specifically

Auto-detection runs when you add a model. The chosen template appears in the Edit dialog dropdown (defaulting to “Auto-detect”). Switching to a specific family value overrides detection.

If you don’t know which family to pick for an unfamiliar model, the model’s Hugging Face page usually names its chat format in the README. Try the closest match - Llama-3-derived models use “Llama”, Qwen-derived use “Qwen”, most other modern models use “ChatML” as a safe default.

Related

• GGUF format - where the template lives in the file
• Weird output - the symptom when template detection is wrong; fix via manual override
• Edit model dialog - where the dropdown lives