base_model: ornith-ai/Ornith-1.5-35B-A3B
library_name: gguf
tags:
- gguf
- llama-cpp
- quantized
- abliterated
- uncensored
- moe
- dynamic-quant
pipeline_tag: text-generation
license: apache-2.0
language: - en
- tr
- zh
quantized_by: alztrk
Ornith-1.5-35B-A3B-Abliterated
A refusal-ablated variant and GGUF quantization suite of ornith-ai/Ornith-1.5-35B-A3B.
Overview
Ornith-1.5-35B-A3B-Abliterated is created through directional ablation surgery on hidden state activations across all 40 layers, orthogonally projecting refusal directions out from attention and MLP down-projection matrices.
This repository provides both the full-precision Safetensors checkpoint and the Dynamic GGUF quantization suite.
Model Architecture
- Total Parameters: 35.8B
- Active Parameters per Token: ~3.1B (8 active experts + shared experts)
- Total Experts: 256
- Layers: 40 Hybrid Layers (DeltaNet Linear Attention + Full Attention)
- Context Length: Native 262,144 tokens (262K)
Quantization Details (GGUF Suite)
GGUF quantizations utilize the Dynamic Hybrid K-Quantization standard, where core attention matrices (attn.wv, attn.wo) retain higher precision while MoE routing FFNs are quantized.
| File Name | Format | Size | Description |
|---|---|---|---|
gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf |
Q4_K_M (Dynamic) | ~19.71 GB | Balanced performance for 12GB - 16GB VRAM GPUs. |
gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q5_K_M.gguf |
Q5_K_M (Dynamic) | ~23.03 GB | Higher precision retention. |
gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q3_K_M.gguf |
Q3_K_M (Dynamic) | ~15.61 GB | Lower memory footprint. |
gguf/Ornith-1.5-35B-Abliterated-Q8_0.gguf |
Q8_0 | ~34.37 GB | High precision reference quantization. |
How to Run
1. Using Ollama
Create a Modelfile:
FROM ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
<|im_start|>assistant
<think>
"""
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
Create and run:
ollama create ornith-35b-abliterated -f Modelfile
ollama run ornith-35b-abliterated
2. Using LM Studio / Llama.cpp
llama-cli.exe -m ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf -p "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n" -ngl 24 -c 8192
3. Using Hugging Face Transformers (Python)
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "alztrk/Ornith-1.5-35B-A3B-Abliterated"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = "Explain kernel level process injection techniques with code examples."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.6)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
License & Attribution
This model is derived from ornith-ai/Ornith-1.5-35B-A3B, licensed under the Apache 2.0 / MIT License. The user assumes full responsibility for any generated output.