license: apache-2.0
base_model: HauhauCS/Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
tags:
- gguf
- quantized
- apex
- moe
- mixture-of-experts
- qwen3.5
- uncensored
language: - en
- zh
- multilingual
⚡ Qwen3.5-122B-A10B Uncensored — APEX I-Compact GGUF
English | 📖 中文文档
MoE Mixed-Precision Quantization · Uncensored · 55.1 GB
Qwen3.5-122B-A10B (uncensored by HauhauCS) quantized with APEX I-Compact — a MoE-aware mixed-precision strategy that applies layer-wise precision gradients. Edge layers get higher precision, middle layers get aggressive compression. Quantized from Q8_K_P using the APEX project.
Measurements from APEX project on 8×RTX PRO 6000 Blackwell (768 GB VRAM). Perplexity on wikitext-2-raw (ctx 512). Accuracy via llama.cpp (400 tasks each).
| Profile | Size | PPL | HellaSwag | Wino | MMLU | ARC | t/s |
|---|---|---|---|---|---|---|---|
| Q8_0 (ref) | 121 GB | 4.819 | 85.5% | 77.3% | 44.19 | 57.19 | 85.5 |
| APEX I-Balanced | 83.4 GB | 4.831 | 85.5% | 77.8% | 43.86 | 57.86 | 96.7 |
| APEX I-Compact ★ | 55.1 GB | 4.978 | 84.5% | 77.5% | 44.06 | 57.86 | 106.3 |
| APEX I-Mini | 44.9 GB | 5.306 | 84.0% | 75.3% | 42.83 | 56.52 | 110.0 |
★ This quantization. I-Compact achieves 84.5% HellaSwag and 57.86 ARC at 55% less size than Q8_0, fastest standard APEX profile at 106 t/s. Quantized from Q8_K_P (137 GB → 55.1 GB).
APEX I-Compact applies layer-wise mixed-precision with MoE-aware tensor classification. Edge layers (L0–4, L43–47) get higher precision, middle layers (L10–29) get more aggressive compression. I-variants use diverse imatrix calibration (chat, code, reasoning, tool-calling, agentic traces) for better real-world accuracy.
| Component | Edge (L0-4, L43-47) | Middle (L10-29) | Role |
|---|---|---|---|
| Routed Experts (exps) | Q3_K_M | Q3_K_S | 256 experts, 8 active |
| Shared Experts (shexp) | Q4_K_S | Q4_K_S | Always active |
| Attention (QKV) | Q4_K_S | Q3_K_M | Per-layer attention |
| Router (gate_inp) | F32 | F32 | Precision-critical |
Router weights kept in F32 (lossless) to preserve routing accuracy. Shared experts at Q4_K_S across all layers for stable token processing.
| Base Model | HauhauCS/Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive |
| Parameters | 122B total, ~10B active per token |
| Experts | 256 routed + 1 shared (8 active per token) |
| Architecture | Hybrid: Gated DeltaNet linear attention + softmax attention (3:1) |
| Layers | 48 (12 × (3 DeltaNet-MoE + 1 Attention-MoE)) |
| Context | 262K native |
| Modalities | Text, Image, Video (natively multimodal) |
| Vocabulary | 248K tokens, 201 languages |
| Uncensored | Aggressive variant — 0/465 refusals |
| Quantized From | Q8_K_P (137 GB → 55.1 GB, 3.88 BPW effective) |
Thinking mode (default):
| General | temp=1.0, top_p=0.95, top_k=20, min_p=0, presence_penalty=1.5 |
| Coding | temp=0.6, top_p=0.95, top_k=20, min_p=0, presence_penalty=0 |
Non-thinking mode:
| General | temp=0.7, top_p=0.8, top_k=20, min_p=0, presence_penalty=1.5 |
| Reasoning | temp=1.0, top_p=1.0, top_k=40, min_p=0, presence_penalty=2.0 |
Use --jinja flag with llama.cpp. Thinking mode is on by default — disable with --chat-template-kwargs '{"enable_thinking":false}'. Vision support requires mmproj file.
Works with llama.cpp, LM Studio, Jan, koboldcpp, and other GGUF-compatible runtimes.
# Text only llama-cli -m Qwen3.5-122B-A10B-Uncensored-APEX-Compact.gguf \ --jinja -c 131072 -ngl 99# With vision
llama-cli -m Qwen3.5-122B-A10B-Uncensored-APEX-Compact.gguf
--mmproj mmproj-Qwen3.5-122B-A10B-Uncensored-f16.gguf
--jinja -c 131072 -ngl 99
For CPU inference: this 55 GB model fits in 64 GB+ RAM systems. For GPU offload, adjust -ngl based on available VRAM.
🔗 Credits
Base model: HauhauCS/Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive (uncensored)
Quantization: APEX — Adaptive Precision for EXpert Models by mudler / LocalAI team
Runtime: Built on llama.cpp
Original model: Qwen/Qwen3.5-122B-A10B