base_model: unsloth/llama-3.2-1b-instruct-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- heretic
- uncensored
- decensored
- abliterated
datasets: - Undi95/Capybara-ShareGPT
license: llama3.2
language: - en
This is a decensored version of joelewing/Llama-3.2-1B-Instruct-Capybara, made using Heretic v1.0.1
Abliteration parameters
| Parameter | Value |
|---|---|
| direction_index | 12.73 |
| attn.o_proj.max_weight | 1.49 |
| attn.o_proj.max_weight_position | 9.36 |
| attn.o_proj.min_weight | 0.65 |
| attn.o_proj.min_weight_distance | 4.53 |
| mlp.down_proj.max_weight | 1.14 |
| mlp.down_proj.max_weight_position | 12.76 |
| mlp.down_proj.min_weight | 1.02 |
| mlp.down_proj.min_weight_distance | 4.68 |
Performance
| Metric | This model | Original model (joelewing/Llama-3.2-1B-Instruct-Capybara) |
|---|---|---|
| KL divergence | 0.03 | 0 (by definition) |
| Refusals | 3/100 | 77/100 |
Llama-3.2-1B-Instruct-Capybara
This model is a finetune of Llama 3.2 1B on the Capybara dataset.
- Developed by: joelewing
- License: Llama 3.2 Community License Agreement
- Finetuned from model : unsloth/llama-3.2-1b-instruct-bnb-4bit
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
