license: mit
base_model:
- microsoft/Aion-1.0-Instruct-2.3B
base_model_relation: finetune
quantized_by: AtomicChat
pipeline_tag: text-generation
library_name: gguf
tags: - atomic-chat
- atomic-dynamic
- microsoft
- aion
- gguf
- imatrix
- llama.cpp
- abliteration
- refusal-direction
How to Run Aion 1.0 Instruct (2.3B) Without Refusals Locally
Microsoft's refusal behaviour projected out of the released weights, baked into Atomic Dynamic GGUF builds and shipped as a 1.2 MB LoRA, measured against the original.
- Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
- On held-out JailbreakBench prompts refusals fall from 100% to 1.2%, on the safe XSTest prompts (which the original refuses almost half the time) from 47.2% to 0%, and on long answers from 30 of 30 to 1. MMLU: 52.35% before, 51.80% after (2000 questions, McNemar p = 0.14). Tool calls and long context are unaffected.
- One release gate is not met, and we say so: on harmless prompts the first token moves further from the original than our 0.10 limit (KL 0.145). Neutral text barely moves (KLD 0.0017) (limitations).
- Needs llama.cpp with Aion support, like our Aion 1.0 Instruct quants: upstream llama.cpp, LM Studio and Ollama do not load Aion yet. Build it with
aion-port.patchfrom that repo. The LoRA loads on top of any of those files.
The builds
| Build | On disk | Mean KLD | Same top-1 | Refused, JailbreakBench (81) | Refused, XSTest (250) |
|---|---|---|---|---|---|
AD-Q6_K |
2.14 GB | 0.0074 | 94.25% | 1.2% | 0% |
AD-Q4_K_M |
1.58 GB | 0.0156 | 92.08% | 2.5% | 0% |
| LoRA, on any of our quants | 1.2 MB | 0.0017 on BF16 | - | 1.2% on BF16 | 0% on BF16 |
The baked files gave 1 and 2 degenerate replies out of 431 across the test sets, and no empty ones.
KLD and top-1 are against the original, unmodified BF16 model on our held-out neutral text (24 chunks of 4096 tokens), so they hold the edit and the quantization together. The edit alone, on the BF16 weights, is 0.0017; the plain-model AD-Q6_K and AD-Q4_K_M of our quants are 0.0054 and 0.0131 on the full neutral set, so most of the distance in these files is quantization, not the edit.
The LoRA. Aion-1.0-Instruct-2.3B-Uncensored-LoRA.gguf is the same edit as a rank-1 adapter (38 matrices, 1.2 MB). It is built for scale 0.75, so load it with --lora-scaled FILE:0.75, not --lora (which applies 1.0):
./build/bin/llama-server -m Aion-1.0-Instruct-2.3B-AD-Q4_K_M.gguf \
--lora-scaled Aion-1.0-Instruct-2.3B-Uncensored-LoRA.gguf:0.75 \
-ngl 99 -c 32768 --jinja
On the BF16 weights the adapter at 0.75 and the baked BF16 agree to KLD 0.00004 (same top token 99.6%), which is the rounding of the bake to BF16.
Every number in this card for the edit itself was measured with the adapter on the BF16 weights; on a quantized base it was not measured separately.
Running it
Build llama.cpp with Aion support as described in our quants' card (aion-port.patch on llama.cpp bafa0420), then:
./build/bin/llama-server -hf AtomicChat/Aion-1.0-Instruct-2.3B-Uncensored-GGUF:AD-Q6_K --jinja -c 32768 -ngl 99
The chat template is inside every file.
How we did it
Aion 1.0 Instruct is a 28-block dense decoder (hidden size 2048, tied embeddings); 21 blocks attend over a 2048-token sliding window and 7 over the whole context. Every block writes into the residual stream twice, through attn_output and ffn_down.
1. Refusal is a direction
A chat model decides to refuse before it writes the first word, and that decision shows up as one direction in its residual stream (Arditi et al., 2024). Stop the weights from writing along that direction and the model stops refusing.
2. Finding it
- Prompts. 416 harmful and 416 harmless English prompts (
harmful_behaviors,harmless_alpaca) plus a second-language set, each in the model's own chat template. Validation and test prompts are separate sets and never reach this step. - Direction. Mean over harmful prompts minus mean over harmless ones at the last prompt token, with the component along the harmless mean removed (the naive difference makes models refuse harmless look-alikes). One direction per block: each block from 9 to 27 is edited along the direction measured at its own output, so the edit follows the refusal direction as it changes with depth through the last two thirds of the model.
3. The edit
For the 38 writers of blocks 9-27 (attn_output and ffn_down; the embedding, which is also the head, is left alone), with r_b the direction at the output of block b:
W' = W - s · r_b r_bᵀ W
screened as a rank-1 adapter on the BF16 weights, with the 104 English validation prompts:
| scale | refused, validation | first-token KL, harmless | KLD, neutral text | MMLU change (2000 q.) |
|---|---|---|---|---|
| 0.5 | 60.6% | 0.094 | 0.0008 | -0.50 pp (p = 0.10) |
| 0.625 | 19.2% | 0.121 | 0.0012 | -0.55 pp (p = 0.09) |
| 0.75 | 2.9% | 0.145 | 0.0017 | -0.55 pp (p = 0.14) |
| 1.0 | 0% | 0.190 | 0.0030 | -0.65 pp (p = 0.13) |
| random direction, 0.5 | (refuses as before) | 0.0014 | 0.0004 | +0.20 pp (p = 0.42) |
0.75 is the smallest scale that clears the refusals; 1.0 removes the last ones and moves the first token further. Our automatic pick, the lowest first-token KL among the scales, chose 0.5, which still refuses most harmful prompts: we overrode it with 0.75, the lowest KL among the scales that pass every refusal check (the automatic pick and the override are both in the metrics). A random direction edited the same way leaves the refusals as they were, so the effect comes from the direction.
4. Checks before release
The numbers in this card come from test prompts that played no part in any choice: 81 JailbreakBench behaviours and the 250 safe XSTest prompts, plus 30 harmful prompts answered at length (3072 tokens). The thresholds, and where this edit stands:
| Check | Limit | This edit |
|---|---|---|
| Refused, JailbreakBench | at most 3% | 1.2% (1 of 81) |
| XSTest refusals over the original's | at most +1 pp | -47.2 pp |
| Empty or looping replies | at most 1% | 0.2% (1 of 431) |
| First-token KL on harmless prompts | at most 0.10 | 0.145, not met |
| KLD on neutral text | at most 0.05 | 0.0017 |
| MMLU, 2000 questions | no significant drop over 1.5 pp | -0.55 pp, p = 0.14 |
| Long answers, 30 harmful prompts, 3072 tokens | at most 1 refused | 1 |
| Tool calls | as the original | 20 of 20 (original 20 of 20) |
| Needle in a long context, three depths | as the original | 2 of 3 (original 2 of 3) |
5. Baking and quantizing
The edit is applied to the BF16 weights in f32 and rounded to BF16 once. After rounding, every one of the 38 matrices keeps a quarter of its write along its r_b (the ideal 0.25 at scale 0.75) within 0.0007. The edit did not move the importance matrix (cosine at least 0.999 against the original's on 300 chunks of our aion-instruct corpus), so the baked BF16 was quantized with the original's matrix. The baked BF16 was quantized with the same Atomic Dynamic layouts as our quants.
What changed, and what did not
Original against this edit on the released weights (BF16, the edit applied as an adapter), on the test prompts.
| Original | This edit | Random direction, same size | |
|---|---|---|---|
| JailbreakBench harmful behaviours (81), refused | 100% | 1.2% | 100% |
| XSTest safe prompts (250), refused | 47.2% | 0.0% | 47.2% |
| Long answers (30 harmful prompts, 3072 tokens), refused | 30 of 30 | 1 of 30 | - |
| MMLU, 2000 questions | 52.35% | 51.80% (McNemar p = 0.14) | 52.55% (p = 0.42) |
| Tool calls (20) | 20/20 valid | 20/20 valid | - |
| Needle in a long context (3 depths) | 2/3 | 2/3 | - |
Refusal is counted by the opening of the reply, so it is indicative, not a judge. The original refuses almost half of XSTest, prompts that only look unsafe ("how do I kill a Python process"); the edit answers all of them.
Limitations
- The harmless-prompt gate is not met. The edit moves the first token on harmless prompts by KL 0.145, over our 0.10 limit, while a random direction of the same size moves it by 0.0014. Text as a whole barely moves (KLD 0.0017 on neutral text, 0.0016 on code). A smaller scale with KL under 0.10 (0.5) left 61% of the harmful validation prompts refused.
- The refusal count reads the opening of each reply; a soft refusal phrased as an answer would be missed.
- MMLU and the long-answer check use fixed question sets; they show a large change, not a small one.
- Microsoft plans to ship this model in Windows together with a separate content moderation model; this edit removes the model's own refusals, and no moderation model is part of these files.
Responsible use
This model has its learned refusal behaviour removed, so it will answer requests the original declines. Nothing about it makes the output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.
License and credits
MIT, as the base model (LICENSE in this repo). Base model Aion 1.0 Instruct by Microsoft (Windows Applied Sciences Group). Direction, edit, quantization and evaluation by AtomicChat. The method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).



