license: other
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags: - moe
- uncensored
- abliterated
- yasha
- gla
Yasha-8B-Abliterated
Base abliterated release. Trained on GLA architecture with MoE 2/16 and ~240K multi-domain samples.
Features
- Abliterated: Orthogonal refusal projection removed from all linear layers
- MoE 2/16: 2 active experts per token, 16 total
- GLA: Gated Linear Attention — O(1) recurrent state, infinite context capability
- Partial RoPE (50%) + YaRN 8x scaling
- Uncensored: No refusal, no guardrails, no alignment filtering
Details
| Param | Value |
|---|---|
| Parameters | ~12.8B total, ~8B active |
| Layers | 80 |
| Hidden | 2048 |
| Heads | 8 × 128d |
| Experts | 16 (top-2) |
| Vocab | 262K |
| Context | 128K native, 1M with YaRN |
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("BeheraBoi/yasha-8b-abliterated")
tokenizer = AutoTokenizer.from_pretrained("BeheraBoi/yasha-8b-abliterated")