license: llama3.2
base_model: DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored
tags:
- llama
- llama-3
- gguf
- quantized
- abliterated
- uncensored
language: - en
pipeline_tag: text-generation
library_name: gguf
Llama-3.2-3B-Instruct-heretic-ablitered-uncensored — GGUF Quantizations
GGUF quantizations of DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored, an abliterated and uncensored version of Meta's Llama 3.2 3B Instruct model. Refusal mechanisms have been removed while preserving the model's general instruction-following capabilities.
Available Quantizations
| File | Bits | Approx Size | Use Case |
|---|---|---|---|
model_f16.gguf |
16 | ~6.4 GB | Maximum quality, reference |
model_q8_0.gguf |
8 | ~3.4 GB | Near-lossless, recommended if RAM allows |
model_q6_k.gguf |
6 | ~2.6 GB | High quality, good balance |
model_q5_k_m.gguf |
5 | ~2.3 GB | Great quality, recommended |
model_q5_k_s.gguf |
5 | ~2.2 GB | Slightly smaller Q5 variant |
model_q4_k_m.gguf |
4 | ~2.0 GB | Best quality/size ratio ⭐ |
model_q4_k_s.gguf |
4 | ~1.9 GB | Smaller Q4 variant |
model_q3_k_l.gguf |
3 | ~1.7 GB | Low RAM, acceptable quality |
model_q3_k_m.gguf |
3 | ~1.6 GB | Lower RAM |
model_q3_k_s.gguf |
3 | ~1.5 GB | Minimal RAM |
model_q2_k.gguf |
2 | ~1.3 GB | Extreme compression, lowest quality |
VRAM / RAM Requirements
| Quant | RAM Required | Fits On |
|---|---|---|
| F16 | ~7 GB | 8 GB GPU / 16 GB RAM |
| Q8_0 | ~4 GB | 6 GB GPU / 8 GB RAM |
| Q6_K | ~3 GB | 4 GB GPU / 8 GB RAM |
| Q5_K_M | ~2.5 GB | 4 GB GPU / 8 GB RAM |
| Q4_K_M | ~2.2 GB | 4 GB GPU / 6 GB RAM |
| Q3_K_M | ~1.8 GB | CPU / 4 GB RAM |
| Q2_K | ~1.5 GB | CPU / 4 GB RAM |
Usage
llama.cpp (CLI)
./llama-cli \
-m model_q4_k_m.gguf \
-p "You are a helpful assistant." \
-cnv \
--temp 0.7 \
-c 4096
llama.cpp (Server)
./llama-server \
-m model_q4_k_m.gguf \
--port 8080 \
-c 4096
llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="model_q4_k_m.gguf",
n_ctx=4096,
n_threads=8,
)
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Tell me about yourself."}
],
temperature=0.7,
max_tokens=512,
)
print(response["choices"][0]["message"]["content"])
Ollama
ollama run hf.co/tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF:Q4_K_M
LM Studio
Search for tinyopsec/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored-GGUF in the LM Studio model browser and download your preferred quant.
About the Original Model
DavidAU/Llama-3.2-3B-Instruct-heretic-ablitered-uncensored is based on Meta's Llama 3.2 3B Instruct with abliteration applied — a technique that surgically removes refusal behavior by identifying and neutralizing the model's "refusal direction" in the residual stream. The result is a model that follows instructions without built-in content restrictions, while maintaining the original's language understanding and reasoning quality.
Key characteristics:
- Architecture:
LlamaForCausalLM - Parameters: ~3 billion
- Context length: 128K tokens
- Instruction-tuned, chat-optimized
- Abliterated — no refusal behavior
- Language: English
Recommended Quant
Q4_K_M — best balance of quality and size for most users. Use Q5_K_M or Q8_0 if you have extra RAM/VRAM.
Credits
- Original model: DavidAU
- Base model: meta-llama/Llama-3.2-3B-Instruct
- Quantization: tinyopsec using llama.cpp