license: apache-2.0
task_categories:
- text-generation
language: - en
size_categories: - n<1K
tags: - atomic-chat
- gguf
- llama.cpp
- abliteration
- refusal-direction
- ternary
- bonsai
- metrics
configs: - config_name: refusal_sweep
data_files: refusal_sweep.csv
default: true - config_name: mmlu
data_files: mmlu.csv - config_name: direction_rows
data_files: direction_rows.csv
Ternary-Bonsai-2-2B-Uncensored-LoRA-GGUF-metrics
Measurements behindAtomicChat/Ternary-Bonsai-2-2B-Uncensored-LoRA-GGUF,
the rank-1 refusal-ablation adapter for PrismML's Ternary Bonsai 2 2B pack (PTQ1_0).
Part of the Ternary Bonsai 2 Uncensored collection.
Aggregates only. Prompt text is not redistributed (the sources are named in the model card)
and model outputs are not published.
| config | harmful: refused | empty/degenerate (of 208) | harmless: refused |
|---|---|---|---|
base |
87.5% | 0 | 0% |
abl:1 |
0% | 0 | 0% |
abl:1.5 |
0% | 0 | 0% |
MMLU (n=300): 31.7% base, 34.3% at scale 1.
| file | rows | what |
|---|---|---|
refusal_sweep.csv |
6 | refusals, empty, degenerate and truncated replies per strength and split |
mmlu.csv |
2 | MMLU accuracy, base and scale 1 |
direction_rows.csv |
25 | per-layer statistics of the estimated direction |
summary.json, mmlu.json, own_proj.json |
- | the same numbers as the pipeline wrote them |
refusal_sweep.csv
104 harmful + 104 harmless held-out prompts, greedy, 256-token budget, thinking off, identical
seed and system prompt across configurations, run on fork release prism-b10777-e3de715. abl:1 is the
adapter at scale 1. refusal_rate_of_valid counts refusals among replies that are neither
empty nor degenerate, so a broken reply is never read as compliance. truncated counts
replies that hit the token budget: an answer runs longer than a refusal, so it rises with
the adapter.
Refusal detection is a rule-based opening-phrase match: indicative, not a judge. A reply
that answers and then adds a disclaimer counts as compliance.
mmlu.csv
300 questions sampled with seed 0 from cais/mmlu, single letter forced by a root ::= [A-D]
grammar, thinking off. At n=300 the standard error is about 2.5-2.7 points.
direction_rows.csv
Row 0 is the embedding output, row L the residual stream entering block L. Per row: AUROC
and Cohen's d separating harmful from harmless on the held-out split, and zero_at - where
ablation puts a prompt on the axis from the harmless mean (0) to the harmful mean (1).zero_at near 0 is what keeps ablation from inducing refusals on ordinary questions. The
adapter uses row 16, the one with the largest Cohen's d (used_by_adapter = 1).
own_proj.json also carries a cos_orca column. At this size it was computed against a
placeholder vector, not a real reference direction (OrcaRouter published one for the 27B
only), so it carries no information and is left out of the CSV.
Reproduction
The recipe is in the model card.