license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags: - magnum
- mistral-nemo
- gguf
- abliterated
- roleplay
- creative-writing
- conversational
- text-generation
Magnum v4 12B Abliterated
An abliterated GGUF release of Magnum v4 12B by Anthracite, focused on natural prose, creative writing, roleplay, and open-ended conversation.
Magnum is a family of language models developed to capture the prose quality associated with Claude 3 Sonnet and Opus. This release builds on those qualities while applying Heretic-based abliteration to reduce learned refusal behavior.
The goal is to retain what makes Magnum enjoyable to use for writing and conversation, with fewer unnecessary refusals. This is an experimental modification, and its behavior may differ from the original model.
Downloads
| File | Format | Approx. size |
|---|---|---|
magnum-v4-12b-heretic-Q5_K_M.gguf |
Q5_K_M | 8.2 GB |
magnum-trial99-bf16.gguf |
BF16 | 23 GB |
Recommended: Q5_K_M offers a practical balance between file size and weight precision for local inference.
The BF16 version is provided for users who want a higher-precision reference or wish to create their own quantizations. It requires substantially more memory.
What is Magnum?
Magnum v4 12B is a 12-billion-parameter, English-language text generation model from Anthracite, fine-tuned from Mistral Nemo Instruct 2407.
Unlike models primarily marketed around coding or benchmark performance, Magnum's design emphasizes prose quality and conversational expression. Its training draws on writing-oriented and conversational datasets, with the aim of producing text that feels more natural, expressive, and engaging.
What it is good for
- Creative writing: stories, descriptive prose, dialogue, and narrative development.
- Roleplay: character-driven interactions, collaborative storytelling, and immersive conversations.
- Conversational use: open-ended discussions and natural back-and-forth exchanges.
- Writing assistance: brainstorming, rewriting, expanding ideas, and exploring different writing styles.
- Instruction-based generation: responding to prompts that call for structured or free-form text.
These describe the intended strengths of the underlying Magnum model. Individual results will vary with prompts, sampling settings, and the particular changes introduced by this release.
What does abliteration change?
Abliteration is a model-weight modification technique intended to suppress certain learned refusal behaviors. This release was produced using Heretic, which searches for ablation parameters that reduce refusal behavior while limiting divergence from the reference model.
The intention is to make Magnum more willing to engage with a broader range of prompts, including prompts that the original model might decline. It does not guarantee that every refusal is removed, nor does it mean the model has gained new knowledge or capabilities.
This is a modification of an existing model, not a new foundation model or a conventional fine-tune on a new instruction dataset.
Initial evaluation
The selected ablation configuration was evaluated on 100 prompts during the Heretic optimization process.
| Metric | Result |
|---|---|
| Refusals | 4 / 100 |
| Refusal rate | 4% |
| KL divergence | 0.0183 |
These are results from the Heretic evaluation setup, not a universal refusal rate. The measured KL divergence describes divergence under that evaluation's reference and methodology. It is not a direct comparison between the BF16 and Q5_K_M files.
Additional evaluation of quantized-model refusal behavior and BF16-versus-Q5_K_M divergence has not yet been completed.
Quantization
Both files use the GGUF format for compatibility with local inference engines such as llama.cpp.
The BF16 GGUF was used as the source for Q5_K_M quantization. An importance matrix generated from a calibration text corpus was supplied during quantization to help preserve model quality.
Quantization reduces storage and memory requirements, but it can also change output quality, instruction following, and refusal behavior. The Q5_K_M version should not be assumed to behave identically to BF16.
Usage
Install a compatible version of llama.cpp and download the Q5_K_M file. Then run:
llama-cli \
-m magnum-v4-12b-heretic-Q5_K_M.gguf \
-cnv
For the BF16 version, substitute magnum-trial99-bf16.gguf.
The BF16 weights occupy approximately 23 GB on disk and exceed the VRAM capacity of a 16 GB GPU. CPU execution or partial GPU offloading may be necessary. Actual memory use depends on context length, runtime overhead, and inference settings.
Limitations
- Abliteration reduces some learned refusal behavior but does not eliminate all refusals.
- Reduced refusal behavior does not imply greater factual accuracy or reasoning ability.
- The model can produce incorrect information, hallucinations, and inconsistent outputs.
- Quantization can introduce additional behavioral differences.
- No comprehensive evaluation of all capabilities or failure modes has been performed for this variant.
Evaluate the model independently before relying on it for any particular application.
Credits and licensing
- Original model: Anthracite, Magnum v4 12B
- Underlying model: Mistral Nemo Instruct 2407
- Abliteration tooling: Heretic
- GGUF conversion and quantization: llama.cpp
This release is distributed under the Apache License 2.0, subject to the applicable terms and notices for the upstream model and its underlying components. Refer to the included LICENSE file and upstream model repositories for the relevant license information.
Disclaimer
This is an experimental model variant intended for local inference, creative work, and model experimentation. No guarantees are made regarding output quality, factual accuracy, safety, or suitability for a particular purpose. Users are responsible for assessing outputs and complying with applicable license terms and laws.