base_model: prism-ml/Ternary-Bonsai-2-27B-gguf
base_model_relation: adapter
library_name: gguf
license: apache-2.0
license_link: https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf/blob/main/LICENSE
pipeline_tag: text-generation
tags:

• gguf
• llama.cpp
• lora
• abliteration
• refusal-direction
• ternary
• bonsai
• prismml
• atomic-chat

How to Run Ternary Bonsai 2 27B Without Refusals Locally

A 9.7 MB adapter, not a checkpoint. The 5.95 GB ternary pack stays byte-identical and the refusal direction is removed inside the compute graph, at a strength you set per request. The measurements behind this card are public.

• Load it with --lora-scaled adapter.gguf:1.5. Scale 0 gives the published model back, token for token, so one process serves both.
• It needs PrismML's llama.cpp fork - stock llama.cpp and Ollama cannot open the base pack at all.
• Every number on this card was measured inside the running model, not inferred from the adapter file.

Ternary-Bonsai-2-27B-Uncensored-LoRA-GGUF

A rank-1 LoRA adapter that removes a refusal direction from
prism-ml/Ternary-Bonsai-2-27B-gguf
at inference time, built and measured by AtomicChat. The base pack stays byte-identical;
nothing is requantized. The direction was estimated on the ternary pack itself, not on
the bf16 model it was trained from.

Why an adapter and not a model

W' = W - r (r^T W) is a dense matrix, but these weights are ternary with an FP16 scale
per 128. The edit is about 1.4% of ||W|| against a grid step of 1.7-2.4x a typical
weight, so baking it back into the pack rounds it away. llama.cpp builds a LoRA into the
graph as two extra matmuls and never merges it into the base weights, which is what makes
a rank-1 adapter an exact carrier of the edit at 1.75 bits/weight.

You need PrismML's llama.cpp fork

Stock llama.cpp cannot open the base pack at all: PTQ1_0 (143) and PQ2_0 (142) are
private ggml type ids, and the weights live in a Hadamard-rotated basis. Use
PrismML-Eng/llama.cpp. Built and verified
against tag prism-b10709-9a9394a.

Get the files

hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir .   # or PQ2_0
hf download AtomicChat/Ternary-Bonsai-2-27B-Uncensored-LoRA-GGUF \
    bonsai2-27b-uncensored-own42-lora.gguf --local-dir .

Run it:

llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf \
    --lora-scaled bonsai2-27b-uncensored-own42-lora.gguf:1.5

Or serve it, with the strength as a per-request knob:

llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 --jinja \
    --lora bonsai2-27b-uncensored-own42-lora.gguf --lora-init-without-apply

curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "..."}],
  "lora": [{"id": 0, "scale": 1.5}]
}'

Use scale 1.5. At 1.0 the projection is exact but 14% of AdvBench-style prompts still
refuse; at 2.0 the model starts emitting empty replies. See the table below. Scale 0
reproduces the published model token for token, so one process can serve both behaviours.

Files

129 sites = 64 ffn_down + 48 ssm_out + 16 attn_output + token_embd. Each adapter
carries its provenance in its GGUF metadata (prism.abliterate.*), including the sha256
of the base pack it was derived from.

How the direction was estimated

Difference of means of the residual stream at the last prompt token (chat template
applied, thinking off), 416 harmful prompts from mlabonne/harmful_behaviors against 416
harmless prompts from mlabonne/harmless_alpaca. The held-out test splits were never used
for estimation. Row L of the direction file is the residual stream entering block L;
row 0 is the embedding output.

The naive difference of means over-refuses. It carries a component along the mean
harmless activation (cosine about -0.3), so zeroing the projection moves a harmless prompt
roughly 3.5 sigma toward the harmful cluster. Measured: the uncorrected row-38 direction
made the model answer "I cannot provide information about the fastest methods for illegal
activities" to a question about the fastest land mammal. The shipped directions subtract
that component, which puts an average harmless prompt back where it was (measured on the
held-out split). The uncorrected file is included only for reproduction.

Data provenance

No prompt set is redistributed here - only the recipe, so anyone can rebuild the same
splits from the sources.

Splits are rebuilt by make_directions.py with sampling seed 0 and mmlu_run.py with
sampling seed 0. numpy.random.Generator does not guarantee an identical stream across
numpy releases, so verify you rebuilt the same subsets against these checksums (produced
with numpy 2.5.3):

038bd43f018f66617129f7bf80c5514f...  test_harmful.jsonl
e640e1637beb3283985487412b684cea...  test_harmless.jsonl
9859cc69902a2b5707afe54346034e8f...  train_harmful.jsonl
087080f508babfd93553662fbaffca84...  train_harmless.jsonl

Model outputs on the harmful split are not published; eval/ carries aggregates only.

Evaluation

104 harmful + 104 harmless held-out prompts, greedy, 64-token budget, thinking off, same
seed and system prompt everywhere. Refusal is a rule-based opening-phrase match, which is
indicative, not a judge. An empty or degenerate reply is counted as damage, not as
compliance.

Restricting the intervention to blocks 15-63 is consistently worse than covering every
site, so the lower blocks and the embedding carry part of the behaviour.

MMLU, 500 questions stratified over 57 subjects, answer forced to a single letter by a
root ::= [A-D] grammar, thinking off. Answer-only, so the absolute number sits below
PrismML's published thinking-mode result; what matters here is the difference.

Flips are symmetric, so there is no systematic loss of knowledge even at scale 2. The
damage at scale 2 shows up in open-ended generation on harmful prompts, not in knowledge.

Both packs

The same adapter file applies to PTQ1_0 and PQ2_0 unchanged, and was run on both.
The two packs carry identical tensor names, shapes and prism.hadamard.* metadata; both
decoders match the F16 pack bit for bit; an adapter rebuilt from PQ2_0 is bit-identical to
the shipped one, tensor for tensor. On PQ2_0 the leak along r falls to 3.6-4.0e-6 at
every writer and stays below 1.6e-5 in the residual stream, and all 416 greedy replies of
the refusal evaluation - base and scale 1.5 - are byte-identical to the PTQ1_0 run. The
packing changes how the trits are stored, not what they are.

A note on head order in ssm_out

The fork rotates the activation only for the base matmul and hands the LoRA branch the
raw activation, so A belongs in the unfolded input basis:

writers:     A = P^T (s * H(r^T W_fold)),   B = -r
token_embd:  A = W_fold @ H(s * r),         B = -r

P is the tiled to grouped V-head permutation that the fork applies to ssm_out when
prism.hadamard.gdn_v_grouped is set. An adapter whose ssm_out factors are left in the
checkpoint's (grouped) head order does not ablate those 48 sites. Verified by tapping the
residual writes inside the fork: with this adapter the component along r falls from
about 1e-2 to 3.8e-6 at every site, and the residual stream leak stays below 1.5e-5 across
all 64 blocks.

Limitations

• The F16 pack was not run. It carries the same tensor names, values and rotation metadata,
  so the adapter should apply, but only PTQ1_0 and PQ2_0 were exercised.
• The refusal metric matches opening phrases only. A reply that answers with a disclaimer
  counts as compliance.
• Capability was checked with MMLU alone. No math, code or agentic benchmark.
• The direction comes from one prompt-pair source at one token position. Transfer to other
  phrasings and languages is unmeasured.

Responsible use

This adapter removes a learned refusal direction, so the model will answer requests the
published model declines. It is an inference-control and research artifact. Nothing about
it makes the resulting output safe, correct or lawful, and any deployment needs its own
access controls and policy enforcement.

Credits

Direction estimation, adapter build and evaluation by nik.bogatyrev.

Base model Ternary Bonsai 2 27B by prism-ml,
Apache-2.0, redistributed here in no form - this repository ships only the adapter.

The refusal-direction method follows Arditi et al., Refusal in Language Models Is
Mediated by a Single Direction (2024). The rank-1 LoRA formulation for this pack, and the
residual-writer inventory it targets, follow
OrcaRouter's OrcaBonsai-27B-Uncensored,
Apache-2.0. The directions shipped here are estimated independently on the ternary pack.