license: mit
base_model: ornith-ai/Ornith-1.5-35B-A3B
base_model_relation: finetune
library_name: transformers
pipeline_tag: text-generation
language: [en]
tags: [gguf, llama-cpp, moe, uncensored, abliterated]
Ornith-1.5-35B-A3B uncensored (GGUF)
TL;DR: abliterated Ornith-1.5-35B-A3B (MoE, 3B active) as a GGUF ladder (Q8_0, Q6_K, Q5_K_M, Q4_K_M + mmproj BF16); hard refusals reduced from 586/1120 to 20/1120 (NVFP4 sibling) with the Q4_K_M build measured at 43/1120 on the same 5-dataset cross-eval.
Same abliterated master as the NVFP4 repo, converted with llama.cpp for local use. Built for security research, red-teaming and penetration-testing workflows; it will comply with requests a stock model refuses. Use responsibly.
Facts & figures
| Base model | ornith-ai/Ornith-1.5-35B-A3B |
| Method | abliteration (1 biprojection pass) + llama.cpp GGUF conversion |
| Files | Q8_0 37.8 GB, Q6_K 29.2 GB, Q5_K_M 25.3 GB, Q4_K_M 21.7 GB, mmproj BF16 0.9 GB |
| Context length | 262,144 tokens (served here with 131,072) |
| Hardware used | 2x NVIDIA RTX PRO 6000 Blackwell 96 GB (SM120) |
| Toolchain | llama.cpp (master, CUDA build), transformers 5.12.1 |
Note: the BF16 GGUF (71.1 GB) is not uploaded because Hugging Face rejects single files over 50 GB; use the Q8_0 file for near-BF16 quality or ask for a sharded BF16 upload.
Refusal behavior (measured, hard markers only)
Effective refusal rate on 1120 harmful prompts across 5 datasets (temp 0, hard decline phrases in the first 25 words; naive keyword counts are inflated by compliant-with-disclaimer answers and are not the metric here). The NVFP4 column is the sibling repo's artifact on the identical master; aggressive quantization re-amplifies hedged declines (Q4 column).
| Eval set | Prompts | Baseline hard | NVFP4 hard | Q4_K_M hard |
|---|---|---|---|---|
| JailbreakBench | 100 | 64 | 4 | 6 |
| tulu-harmbench | 320 | 132 | 6 | 10 |
| HarmfulQA | 300 | 84 | 0 | 1 |
| LLM-LAT | 300 | 226 | 6 | 21 |
| mlabonne harmful | 100 | 80 | 4 | 5 |
| TOTAL | 1120 | 586 | 20 | 43 |
Coherence of the base master (measured on the NVFP4 sibling): GPQA-Diamond 58.6% (base model 61.1%), needle-in-haystack 1.0 at 32k/65k/131k, tool-call smoke 1.0, KL(base||ablated) median 0.218 (harmless subset 0.110). The underlying BF16 master is effectively refusal-free on the harvest pool (0.4%).
Run it with llama.cpp (validated)
llama-server -m Ornith-1.5-35B-A3B-uncensored-Q4_K_M.gguf \
--mmproj mmproj-Ornith-1.5-35B-A3B-uncensored-BF16.gguf \
--jinja -ngl 99 -c 131072 --port 30003
Quality & limitations
- Quantization amplifies residual refusals on this model: NVFP4 20/1120, GGUF Q4_K_M 43/1120 on the same eval. For the cleanest behavior use the NVFP4 repo or the Q8_0 file here.
- MTP speculative decoding is not wired in these GGUFs; the conversion carries the language model plus the multimodal projector (mmproj included).
Provenance & reproducibility
Same master as ressl/Ornith-1.5-35B-A3B-uncensored-NVFP4: residual directions from harmful-vs-harmless rounds, norm-preserving biprojection on 120 residual-writing tensors (including the 256-expert fused down_proj, batched). GGUF via convert_hf_to_gguf.py (BF16 + mmproj) and llama-quantize for the K-quants, validated with llama-server on the full cross-eval plus a German coherence smoke.
License & credits
MIT (inherited from Ornith by the Ornith team, credited to ornith-ai).
Abliteration, conversion and validation by Robert Ressl (Hugging Face · Website · LinkedIn · Patreon). Built with llama.cpp, NVIDIA TensorRT Model Optimizer and SGLang.
Support this work: if these models are useful to you, consider supporting on Patreon. A lot of compute and care went into this release; more at ressl.ch.