license: other
license_name: lfm1.0
license_link: https://huggingface.co/LiquidAI/LFM2-24B-A2B/blob/main/LICENSE
base_model: huihui-ai/Huihui-LFM2-24B-A2B-abliterated
tags:
- gguf
- quantized
- apex
- moe
- mixture-of-experts
- liquidai
- lfm2
- hybrid
- abliterated
⚡ Each donation = another big MoE quantized
I host 30+ free APEX MoE quantizations as independent research. My only local hardware is an NVIDIA DGX Spark (122 GB unified memory), enough for ~30-50B-class MoEs, but bigger ones (200B+) require rented compute on H100/H200/Blackwell, typically $20-100 per quant.
If APEX quants are useful to you, your support directly funds those bigger runs.
Huihui-LFM2-24B-A2B-abliterated APEX GGUF
APEX (Adaptive Precision for EXpert Models) quantizations of huihui-ai/Huihui-LFM2-24B-A2B-abliterated, an abliterated (refusal-removed) fine-tune of LiquidAI/LFM2-24B-A2B.
Brought to you by the LocalAI team | APEX Project | Technical Report
Status: unevaluated. No benchmark or quality evaluation was run on these files. The only checks were that each file loads in llama.cpp and generates a short, readable answer.
Files
| File | Size | For |
|---|---|---|
| Huihui-LFM2-24B-A2B-abliterated-APEX-Quality.gguf | 15.28 GB | Quality recipe |
| Huihui-LFM2-24B-A2B-abliterated-APEX-I-Quality.gguf | 15.28 GB | Quality recipe with imatrix |
| Huihui-LFM2-24B-A2B-abliterated-APEX-Balanced.gguf | 17.09 GB | Balanced recipe |
| Huihui-LFM2-24B-A2B-abliterated-APEX-I-Balanced.gguf | 17.09 GB | Balanced recipe with imatrix |
| Huihui-LFM2-24B-A2B-abliterated-APEX-Compact.gguf | 11.07 GB | Compact recipe, smaller files |
| Huihui-LFM2-24B-A2B-abliterated-APEX-I-Compact.gguf | 11.07 GB | Compact recipe with imatrix |
| Huihui-LFM2-24B-A2B-abliterated-APEX-I-Mini.gguf | 9.01 GB | smallest, imatrix only |
I- files use an importance matrix built from diverse calibration data (chat, code, reasoning, tool-calling, agentic traces, Wikipedia). Quality, Balanced and Compact also ship without it. Mini ships only as I-Mini.
What is APEX?
APEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient: edge layers get higher precision, middle layers get more aggressive compression. See the APEX project for details, the technical report and the scripts.
The model
Huihui-LFM2-24B-A2B-abliterated is a Mixture-of-Experts model of about 24B parameters, with about 2B active per token. It has 40 layers (30 convolutional and 10 full attention, hybrid), 64 routed experts with 4 active per token, and 2 dense layers. It is text only. The base model is LiquidAI's LFM2-24B-A2B; huihui-ai removed its refusal behavior with abliteration.
This model is abliterated. It does not refuse requests the way the base model does, so you are responsible for how you use its output. These quants are unevaluated: abliteration and quantization can each change behavior in ways that were not measured here.
Usage
Use a recent llama.cpp build with lfm2moe support. These files were checked with build b12024 (commit ddf48f85c).
llama-cli -m Huihui-LFM2-24B-A2B-abliterated-APEX-I-Balanced.gguf -p "Hello"
Run with LocalAI
local-ai run mudler/Huihui-LFM2-24B-A2B-abliterated-APEX-GGUF@Huihui-LFM2-24B-A2B-abliterated-APEX-I-Balanced.gguf
License
The base model, LiquidAI/LFM2-24B-A2B, is released under the LFM Open License v1.0 (lfm1.0), and so is the huihui-ai fine-tune. These files are derivatives of that model and are covered by the same license, not by Apache-2.0 or any other license. In short, commercial use is licensed only for organizations with annual revenue under 10 million US dollars; larger organizations need a separate license from Liquid AI. Read the license text for the full conditions on use, commercial use and redistribution before you use or share these files. When you redistribute them, include a copy of the license and keep the attribution to Liquid AI and huihui-ai.
Credits
Base model by Liquid AI. Abliteration by huihui-ai. APEX is brought to you by the LocalAI team. Developed through human-driven, AI-assisted research. Built on llama.cpp.