license: mit
license_link: https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B/blob/main/LICENSE
base_model:
- ornith-ai/Ornith-1.5-35B-A3B
pipeline_tag: text-generation
library_name: transformers
tags: - abliterated
- uncensored
- moe
- mamba
Ornith-1.5-35B-A3B-abliterated
Abliterated (refusal-direction removed) build ofornith-ai/Ornith-1.5-35B-A3B, BF16, 65 GB.
An NVFP4 quantization of this model (20 GB, runs on 2×16 GB GPUs, with benchmarks) is atpottokao/Ornith-1.5-35B-A3B-abliterated-NVFP4-DFlash.
⚠️ Text-only. This is a language-model-only export: it contains no vision tower and no MTP
head, unlike the original release.model_typeisqwen3_5_moe_text,
architectureQwen3_5MoeForCausalLM.
⚠️ Uncensored. Safety refusal behaviour has been deliberately removed. You are responsible
for how you use it.
Method
Classic refusal-direction ablation (orthogonalization), single direction:
| Step | Detail |
|---|---|
| Probe layer | 24 — int(num_layers × 0.6), 40 layers total |
| Samples | 64 harmful + 64 harmless prompts (random.seed(0)), last-token hidden state |
| Direction | d = normalize(mean(harmful) − mean(harmless)) |
| Ablation | For every .o_proj and .down_proj: W ← W − outer(d, dᵀW) |
The refusal direction is projected out of the output space of the attention- and MLP-output
projections, leaving all other weights untouched. Tooling derived fromremove-refusals-with-transformers.
Architecture
Hybrid Mamba + MoE, 40 layers: 30 linear-attention (Mamba-style) layers and 10 full-attention
layers (indices 3, 7, …, 39), 256 experts per layer with a shared expert, ~3 B active parameters.
Context length 262,144.