Ternary-Bonsai-2-27B (Uncensored / Steered Edition)

This repository provides Ternary-Bonsai-2-27B-PQ2_0 coupled with a dedicated Refusal Cancellation Control Vector (refusal_vector_27b.gguf) to bypass alignment refusals, preachy lecturing, and refusal barriers at inference time.

⚡ Key Highlights

• Architecture: Qwen 3.x 27B derivative (64 layers, 4 KV heads).
• Quantization: 1.58-bit Ternary (PQ2_0), weighing only 7.21 GB on disk.
• Full Context: Up to 131,072 tokens (128k context) supported in <10 GB VRAM.
• Max Generation Output: Up to 16,384 tokens.
• Refusal Steering: PCA control vector across all 64 decoder layers that subtracts the refusal subspace at runtime ($\alpha = -1.0$).

❓ How Refusal Removal Works on This Model

Unlike traditional FP16 models where weights are directly projected and re-quantized, ternary models use discrete ${-1, 0, 1}$ bitplanes that cannot be permanently altered without destroying the integer quantization kernels.

Instead, this repository uses native llama.cpp Control Vector Steering (--control-vector-scaled refusal_vector_27b.gguf:-1.0). At every generation step, the refusal direction is dynamically subtracted from the residual stream at full float precision.

What this achieves:

• No moralizing lectures: The model ceases outputting preambles like "I cannot assist with...", "As an AI...", or ethical lectures.
• Direct technical analysis: Explains vulnerabilities, reverse engineering, security mechanics, and sensitive topics objectively.
• Zero weight degradation: Preserves the original 1.58-bit ternary precision and reasoning capabilities without perplexity collapse.

🚀 Quickstart: Run with llama.cpp

1. Download Files

huggingface-cli download Secondchancee/Ternary-Bonsai-2-27B-Steered --local-dir ./bonsai-27b
cd ./bonsai-27b

2. Run API Server (OpenAI & Anthropic Compatible)

llama-server   -m Ternary-Bonsai-2-27B-PQ2_0.gguf   --control-vector-scaled refusal_vector_27b.gguf:-1.0   -ngl 99   --flash-attn on   --cont-batching   --cache-type-k q4_0   --cache-type-v q4_0   -c 131072   -n 16384   --port 8000   --host 0.0.0.0

3. Run in Terminal (CLI Mode)

llama-cli   -m Ternary-Bonsai-2-27B-PQ2_0.gguf   --control-vector-scaled refusal_vector_27b.gguf:-1.0   -ngl 99   -c 8192   -p "<|im_start|>user\nExplain the low-level mechanics of buffer overflows.<|im_end|>\n<|im_start|>assistant\n"

⚙️ Recommended Settings

• Steering Strength (α): -1.0 (optimal balance). Values between -0.8 and -1.2 maintain high reasoning quality while suppressing refusals.
• Reasoning Effort: If using client tools like Cursor, Roo Code, or Cline, set reasoning_effort to medium or high (xhigh).
• Temperature: 0.6 – 0.7.

📜 Citation & Credits

• Base Quantized Model by Prism ML
• Steering methodology based on Arditi et al. (2024), "Refusal in Language Models Is Mediated by a Single Direction"