base_model: microsoft/Fara-7B
library_name: transformers
license: other
pipeline_tag: text-generation
tags:
- abliteration
- refusal-removal
- uncensored
- research
- qwen2_5_vl
- orthogonalization
Fara-7B Abliterated v2
A refusal-direction-orthogonalized variant of microsoft/Fara-7B (Qwen2.5-VL based).
Built using:
Method
Using harmful + harmless probe sets, residual-stream activations were extracted across layers 0–27 to identify the strongest refusal direction.
Best layer:
- 13
Orthogonalization was applied in fp32 to:
embed_tokens- every
self_attn.o_proj - every
mlp.down_proj
Total modified tensors:
- 57
Formula:
W ← W - r rᵀ W
Results
Held-out harmful evaluation set:
- Original Fara-7B: 5/160 compliance (~3.1%)
- Abliterated v2: 158/160 compliance (~98.75%)
Held-out refusal probe:
- Before: 155/160 refusals
- After: 2/160 refusals
Notes
- fp32 surgery used to avoid precision issues from v1
- edits applied only to the language tower
- held-out evaluation set was separate from the layer-selection probe set
Research artifact only. Use responsibly and follow upstream Fara/Qwen license terms.