library_name: gguf
license: mit
pipeline_tag: text-generation
tags:
- gguf
- moe
- hybrid
- reasoning
- abliterated
- quantization
- bailingmoe3
- inclusionAI
base_model: - inclusionAI/Ling-3.0-flash
- SC117/Ling-3.0-flash-abliterated-APEX-GGUF
base_model_relation: quantized
Ling-3.0-flash-abliterated — GGUF Quants
Standard static quants (no imatrix) + APEX mixed-precision quants of SC117's abliterix v6 trial 21 merge.
Original model: SC117/Ling-3.0-flash-abliterated-APEX-GGUF
Base model: inclusionAI/Ling-3.0-flash
Abliteration by: abliterix v6 trial 21
Architecture: bailingmoe3 (124B total, 5.1B active)
License: MIT
Files
Standard Static Quants (llama-quantize, no imatrix)
| File | Type | Approx Size |
|---|---|---|
*-Q2_K.gguf |
Q2_K | ~40 GB |
*-Q3_K_S.gguf |
Q3_K_S | ~50 GB |
*-Q3_K_M.gguf |
Q3_K_M | ~55 GB |
*-Q3_K_L.gguf |
Q3_K_L | ~60 GB |
*-Q4_K_M.gguf |
Q4_K_M | ~70 GB |
*-Q5_K_M.gguf |
Q5_K_M | ~90 GB |
*-Q6_K.gguf |
Q6_K | ~108 GB |
*-Q8_0.gguf |
Q8_0 | ~140 GB |
APEX Mixed-Precision Quants
| File | Base | Experts | Shared | Attn |
|---|---|---|---|---|
*-APEX-Quality.gguf |
Q6_K | edge Q6_K / mid iq4_xs | Q8_0 | Q6_K |
*-APEX-Balanced.gguf |
Q6_K | edge Q6_K / mid Q5_K | Q8_0 | Q6_K |
*-APEX-Compact.gguf |
Q4_K_M | edge Q4_K / mid Q3_K | Q6_K | Q4_K |
All APEX configs: routers Q8_0, MTP (blk.42) Q8_0. No imatrix required.
APEX Mini (iq2_s mid) excluded — requires imatrix for quantization.
Usage
Requires llama.cpp with PR #26608 (bailingmoe3 support, merged b10472+).
llama-server -m
Sampling: temperature 0.6, top_p 0.95, top_k 20
Credits
- Model: inclusionAI/Ling-3.0-flash
- Abliteration: SC117 using abliterix v6 trial 21
- APEX quantization method: mudler/apex-quant
- GGUF quants: This repo (quantization only, no model modifications)