license: apache-2.0
base_model: 0xKitkat/Ornith-1.5-35B-A3B-Uncensored
library_name: gguf
tags:

• text-generation
• gguf
• quantizer
• autoround
• architecture-aware
• moe
• mixture-of-experts
• uncensored
• 35b

Ornith-1.5-35B-Uncensored-YMQ-MTP-GGUF

Source Model: 0xKitkat/Ornith-1.5-35B-A3B-Uncensored

⚖️ An Architecture-Aware, AutoRound-Inspired MoE Mixed Precision Layout

This repository features advanced, custom architecture-aware quantizations of Ornith-1.5-35B (Uncensored) processed directly from official raw BF16 source files using the custom YMQ-Compiler (v2.0) log-space framework.

These builds natively preserve the delicate Mixture-of-Experts routing topology and utilize high-context optimization parameters tailored for demanding local agent execution environments (such as RooCode/Aider).

📊 Quantization Preset Tier Details

📉 Perplexity Evaluation Metrics (WikiText-2)

The following metrics show the mathematical quality preservation of the YMQ-Compiler log-space cluster analysis compared to standard linear quantization layouts. Tested natively via llama-perplexity over a 4096 context window using the official WikiText-2 test corpus.

💡 The MoE Compression Breakthrough: YMQ vs. Standard Quants

Standard quantization pipelines act like a blunt hammer. They apply a uniform, flat bit-depth across every layer of the model, which completely breaks the delicate routing paths of Mixture-of-Experts (MoE) architectures.

Independent local testing using the official llama-perplexity harness exposes the massive optimization gap between standard, flat layouts and the architecture-aware YMQ-Compiler:

• Standard Flat Q4_K_M (~21.0 GB):

  • Hits a high 13.4757 perplexity score on the test corpus.
  • Starves the core attention entry channels and native MTP speculator tracking paths of bit-depth resolution.
  • Model experiences severe tracking fatigue under stress.

• YMQ-Compiler M Preset (~16.0 GB):

  • Scores a spectacular 12.4310 perplexity score on the exact same corpus.
  • Surgically isolates and insulates critical logic entry gates behind high-fidelity shields (Q5_K / Q6_K).
  • Heavily compresses background expert layers into non-linear matrices.
  • Saves a massive 5 Gabytes of VRAM while delivering a drastic leap in logical reasoning clarity.

The Practical Result

The practical result is an elite, filter-free developer build that completely eliminates late-stage context amnesia bugs.

• Flat, uniform community quants suffer from endless reading loops and failed edits past 30k tokens.
• YMQ-M and S presets stream massive 200k+ context windows completely in VRAM.
• Blistering multi-token speculation speeds (80–120 t/s).
• Zero loop failures!

⚖️ YMQ vs. Uniform Quantization (The AutoRound Philosophy)

Standard quantization pipelines apply a blunt, uniform bit-depth across every single layer in a model. This blunt approach completely collapses the delicate routing structures of large Mixture-of-Experts (MoE) architectures, starving critical logic anchors of necessary precision while bloating file sizes with idle parameters.

The YMQ-Compiler implements a post-training optimization philosophy similar to advanced weight-tuning frameworks like Intel's AutoRound:

• Targeted Bit Isolation: Instead of applying a flat matrix mask, YMQ operates strictly in Log-Space. It surgically identifies the core attention entry lanes and high-leverage routing networks, locking them behind heavy Q5_K and Q6_K high-fidelity safety shields.
• Expert Layer Flattening: It takes the massive pool of background expert weights (ffn_*_exps) and compresses them aggressively using dense, non-linear grids (IQ2_XS and IQ4_NL). Because these background parameters make up the majority of the file footprint but are rarely active at the same time, the compiler shaves off gigabytes of background noise without breaking the model's main train of thought.

The result is a highly stable, custom mixed-precision portfolio that matches the low perplexity and high instruction clarity of premium optimized configurations, while allowing 16GB and 24GB single-GPU setups to stream massive context windows with absolute structural peace of mind!

🛠️ The YMQ Compilation Architecture

Standard quantization pipelines treat network tensors like a flat dataset, applying destructive blanket low-bit compression to delicate tracking networks. The YMQ-Compiler solves high-context logic decay by parsing model files dynamically via an automated, multi-tiered protection matrix:

1. Log-Space Gap Detection Clustering: Instead of flat percentage thresholds, the engine computes statistical cluster variances in log-space, successfully isolating intermediate logical reasoning spikes and elevating them to stable non-linear 4-bit (IQ4_XS) formats, while compressing idle fact-storage layers to aggressive 2-bit baselines.
2. Fading Boundary Tapering: Recognizes the extreme fragility of initial token entry data vectors, forcing an input wave cushion (L00=IQ4_NL → L01=IQ4_XS → L02=IQ3_XXS) that gradually stabilizes parameters before hitting the fallback pools.
3. Dedicated Gate Insulation: Hard-shields volatile parallel Transformer Multi-Head Attention and MoE expert routing paths, keeping context tracking perfectly noise-free.
4. Asymmetric Vocabulary Shielding: Fixes tied-weight boundary errors by mapping the final logit classification exit heads to robust configurations to completely eliminate formatting loops and API tag leakage under deep contexts.

🚀 Recommended Runtime Parameters (llama.cpp / llama-server)

$./llama-server -m Ornith-1.5-35B-Uncensored-YMQ-M.gguf -ctk q8_0 -ctv q4_0 --ctx-size 245760 --mmproj mmproj-Ornith-1.5-35B-Uncensored-Q6_K.gguf \
  --n-cpu-moe 5 --timeout 36000 --checkpoint-min-step 2048 --ctx-checkpoints 4 --spec-type draft-mtp --spec-draft-n-max 2 \
  --n-predict -1 --temp 0.6 --top-p 0.95 --top-k 20 --repeat-penalty 1.05 --jinja -fa

🖼️ Vision Projection (--mmproj)

For multimodal vision support, pair these builds with one of the following projection files:

Pass via --mmproj <path-to-file> in your llama-server invocation (see example above).

☕ Support & Future R&D

If the YMQ-Compiler builds saved your context window from collapsing or optimized your active development cycle speeds, consider buying a coffee to fund further low-level optimization research. Your support keeps the server nodes baking future model scales!

👉 Support ZeroDigest Research on ko-fi

📦 Source Framework & Automation Code

The compiler pipeline automation engine, setup thresholds, and structural mapping rules are open-source. To view the implementation details or compile your own custom models natively using this profile layout, visit the official development hub:

👉 GitHub: ZeroDigest / YMQ-Compiler