GGUF - the file format Abliteration reads

Short answer

GGUF is a binary file format that packs the model weights, its architecture description, its tokenizer, and its chat template into one file. Abliteration only accepts GGUF files.

Detailed explanation

GGUF stands for “GPT-Generated Unified Format”. It was designed by the llama.cpp project to be self-contained: everything needed to run the model lives inside one file. No separate config.json, no separate tokenizer.json, no separate template.

A GGUF file starts with a header containing metadata as key/value pairs, then a tensor list, then the actual weights. The metadata section is what makes GGUF powerful - it holds things like general.architecture (llama, qwen2, mistral…), {arch}.context_length (the max context the model was trained on), {arch}.block_count (number of transformer layers), {arch}.embedding_length (hidden size), {arch}.expert_count (for MoE models), and tokenizer.chat_template (the Jinja template for formatting prompts).

Abliteration parses the GGUF header directly on model add - it doesn’t rely on the filename. That’s why a file called my-random-name.gguf still gets correctly identified as a Qwen 2.5 7B with 128K context and Q4_K_M quantization. Everything the app needs to display in the Model info tooltip comes from that header read.

Reading the header is fast (a few KB from the front of the file) - the full model file, which can be 20+ GB, is only touched when you actually load the model for inference.

Why it matters

If you download a model in another format (safetensors, PyTorch .bin, ONNX), Abliteration can’t use it. You need the GGUF conversion - usually posted on Hugging Face by community quantizers like unsloth, bartowski, TheBloke (historical), or the model author themselves.

The fact that GGUF is self-contained is also why chat templates auto-detect correctly in Abliteration: the template is baked into the file, so we don’t need to guess “is this Llama-style or ChatML?”.

In Abliteration specifically

When you drag a GGUF into Abliteration:

1. The GGUF header is parsed (Services/GgufMetadata.cs)
2. n_ctx_train, n_embd, n_layers, expert_count, head_count_kv are extracted
3. These feed the RAM estimate for fit-check
4. Chat template from tokenizer.chat_template is used if present, otherwise a family-based fallback (Llama / Qwen / Gemma / Phi / ChatML) kicks in
5. Quantization is read from general.file_type and used for the ·Q4/·Q8 suffix in model names

If the header is malformed or missing fields, Abliteration falls back to a filename-based guess and shows a warning.

Related

• Chat template auto-detection - how templates come out of GGUF
• n_ctx_train - the trained context ceiling read from the header
• Quantization - read from general.file_type