license: other
license_name: lfm1.0
license_link: LICENSE
base_model: unsloth/LFM2.5-24B-A2B
pipeline_tag: text-generation
tags:
- gguf
- lfm2_moe
- lfm2.5
- moe
- abliterated
- heretic
- nvfp4
LFM2.5-24B-A2B Abliterated - GGUF
GGUF quantizations of a modified, partially refusal-reduced LFM2.5-24B-A2B checkpoint. This repository contains Q4_K_M, Q8_0, and experimental NVFP4 files of the same abliterated model, not separately trained models.
The source is unsloth/LFM2.5-24B-A2B, revision f8919e5dc304fd3c82b194963dc16a1a59f88623, derived from LiquidAI/LFM2.5-24B-A2B. The original tokenizer and always-thinking chat template are retained.
Files
| File | Quantization | Size |
|---|---|---|
LFM2.5-24B-A2B-abliterated-Q4_K_M.gguf |
Q4_K_M | 14.53 GB |
LFM2.5-24B-A2B-abliterated-Q8_0.gguf |
Q8_0 | 25.49 GB |
LFM2.5-24B-A2B-abliterated-NVFP4.gguf |
NVFP4 experts; Q8_0 dense/embedding | 14.02 GB |
Use a recent llama.cpp or LM Studio runtime supporting the lfm2moe architecture. Q4_K_M is a standard mixed-precision K-quant; some tensors intentionally remain Q6_K or F32. It was quantized from a BF16 GGUF converted from the saved abliterated checkpoint, not requantized from Q8_0. No variant has a serving-speed benchmark establishing a particular tokens-per-second rate.
Experimental NVFP4 variant
The NVFP4 file has 114 routed-expert tensors in genuine NVFP4, 107 dense/embedding matrices in Q8_0, and 207 F32 tensors preserved bit-for-bit. It is an experts-only NVFP4 recipe, not an all-NVFP4 checkpoint or a calibrated NVIDIA ModelOpt export.
Quantization used the reference routines exposed by the unmerged llama.cpp PR #26869, pinned to c100a987c7e33f41549add572a3f47d26e306a43, without calibration or an importance matrix. The GGUF file type is 39 (NVFP4); its tokenizer, chat template, model metadata, and tensor dimensions were checked against the BF16 conversion.
It passed exact READY, arithmetic, and JSON-output checks on an NVIDIA GB10 using LM Studio's CUDA 13 runtime 2.58.0, with 4096-token context, greedy decoding, a 128-token thinking budget, and a 256-token output limit. These are three benign smoke checks, not a capability or performance benchmark. This desktop version displays the model's NVFP4 name but shows an unknown quantization badge (?); its inference runtime loaded and ran the file in the checks.
Method and attribution
This is a credited, same-source reproduction of AtomicChat's published late-block directional edit, not a newly discovered refusal direction:
- Direction source: AtomicChat/LFM2.5-24B-A2B-Uncensored-GGUF-metrics, revision
78e022b6d4173524696c434d4c0f593929eb5dc4, arrayenru. - Edited blocks: 14-39 inclusive, strength 1.75, per-layer residual direction row
block + 1, no row normalization. - Edited writers: attention/short-convolution outputs and fused MoE expert down projections in those blocks. The leading dense MLP blocks were not edited by this late-block recipe.
- Engine: bundled Heretic 1.4.0 through the llm-abliterate toolkit, with an architecture-specific fused-MoE adapter.
- All three quantizations were produced from the same saved BF16 edited checkpoint. There was no additional fine-tuning or training for these variants.
Limited evaluation
The following measurements apply to the saved BF16 checkpoint, not full benchmarks of these GGUF quantizations. The source and edited model used matched prompts, native reasoning, greedy decoding, a helpful-assistant system prompt, and a 2048-token generation budget.
| Measurement | Source | Edited BF16 |
|---|---|---|
| Keyword refusals on 16 refusal-eliciting prompts | 16 | 1 |
| Incomplete thinking on those 16 prompts | 0 | 1 |
| Final-answer stage reached on 32 benign prompts | 32 | 32 |
| GSM8K flexible-extraction accuracy, 32 examples | 25/32 | 25/32 |
| IFEval strict prompt-level passes, 32 examples | 27/32 | 25/32 |
The screening evaluator counts incomplete thinking as a failure, giving 2/16 candidate failures: one keyword refusal and one incomplete generation. These are not calibrated semantic refusal or correctness judgments. Reaching a final answer does not establish that the final answer was untruncated. The IFEval pilot shows a two-example regression.
The public recipe's development data may overlap public evaluation banks; these results should not be described as an independently clean final benchmark. Small samples do not establish general capability retention or complete refusal removal.
Q4_K_M and Q8_0 each passed a CPU llama.cpp reload and a benign final-answer check. NVFP4 passed the three CUDA runtime checks described above. Full post-quantization quality, refusal, long-context, tool-use, language, and performance evaluations remain outstanding for all variants. See evaluation_summary.json and SHA256SUMS for the recorded provenance and artifact hashes.
License and modification notice
These are modified derivatives, not official Liquid AI releases. The original LFM Open License v1.0 applies; see LICENSE and NOTICE. Its commercial-use limitation, including the USD 10 million annual-revenue threshold, still applies. This repository does not relicense the original model under Apache or MIT terms.