license: apache-2.0
base_model: dima6312/Flint-40B-A2.5B-abliterated
base_model_relation: quantized
pipeline_tag: text-generation
library_name: gguf
language:
- en
tags: - abliterated
- cybersecurity
- code
- moe
- gguf
- nvfp4
Flint-40B-A2.5B-abliterated (NVFP4 GGUF)
RecursiveSI's Flint 40B-A2.5B with its refusal direction removed, for
security research and coding work, as an NVFP4 GGUF for NVIDIA Blackwell GPUs with llama.cpp CUDA.
Which build
| You run | Download | Size |
|---|---|---|
| llama.cpp, or an app that bundles llama.cpp b11827 or later; any GPU, CPU or Mac | GGUF: Q8_0 is the benchmarked build, Q4_K_M the usual choice |
42 / 24 GB |
| NVIDIA Blackwell GPU (RTX 50 series, RTX PRO 6000, DGX Spark) with llama.cpp CUDA | NVFP4 GGUF (this repo) | 23 GB |
| Apple silicon Mac with RecursiveSI's Rec runtime | MLX 4-bit | 21 GB |
| transformers or SGLang on a GPU server; fine-tuning or making your own quants | BF16 safetensors | 80 GB |
Plan for the file size plus memory for context: about 26 KB per token, so roughly 7 GB at the full 262,144 tokens.
The file Flint-40B-A2.5B-abliterated-NVFP4.gguf (22.9 GB) uses the same tensor layout, types and scales as
RecursiveSI's flint-nvfp4-gguf, built from our BF16 weights.
Results
Short paired check against RecursiveSI's NVFP4 file, both on llama-server (llama.cpp b11827) with the same
flags, Flint's recommended sampling (temperature 1.0, top_p 0.95, top_k 20) and full reasoning, on an RTX PRO 6000
Blackwell. Same items for both models; a quality check fails only on a drop that is significant at 95%.
| Check | RecursiveSI NVFP4 | Abliterated NVFP4 | Result |
|---|---|---|---|
| MITRE refusals (n=50), held out from the search | 22% | 0% | pass |
| MITRE replies with no final answer (still reasoning at 8,192 tokens) | 2 | 12 | info |
| GSM8K (n=40) not significantly worse | 95% | 95% | pass |
| HumanEval+ (n=40) not significantly worse | 90% | 92.5% | pass |
| pp512 tok/s within 3% | 6814 | 6815 | pass |
| tg128 tok/s within 3% | 207.4 | 207.5 | pass |
Heretic's default refusal count, measured on the NVFP4 builds (100 mlabonne/harmful_behaviors test prompts, reasoning off, greedy, 100 tokens): original 96/100, abliterated 55/100. That count flags any reply containing words such as "harmful" or "illegal", including answers that comply; only 1 of the 55 replies opens with a refusal. Meta's CyberSecEval refusal check on the same replies: original 18/100, abliterated 1/100.
The full benchmark set (Q8_0, more items) is on the main card.
Usage
Use a llama.cpp CUDA build with Flint support (b11827 or later). Plan for at least 48 GB of GPU memory, or
unified memory on DGX Spark.
hf download dima6312/Flint-40B-A2.5B-abliterated-NVFP4-GGUF Flint-40B-A2.5B-abliterated-NVFP4.gguf --local-dir .
llama-server -m Flint-40B-A2.5B-abliterated-NVFP4.gguf \
-ngl all -c 262144 -fa on --jinja \
--temp 1 --top-p 0.95 --top-k 20 --min-p 0
Reasoning is optional; when the model reasons, it uses <think>…</think> before its answer. Give long security tasks
a large token budget: on the main checks, 19 of 100 MITRE answers were still reasoning at 8,192 tokens.
Limitations
- The check above is short (n in each row). Refusals are lower, not zero.
- Abliteration removes refusals; it adds no knowledge. Mistakes the original makes, this model makes too.
Responsible use
This model answers requests the original refuses. It is meant for authorized security testing, research, education
and defensive work. You are responsible for how you use it and for following the law and the terms of the systems you
test. Its answers can be wrong; check them before acting on them.
Credits
- Flint and the NVFP4 GGUF layout by RecursiveSI (Apache 2.0). This is a derivative of their weights under the
same license. - abliterix by wuwangzhang1216, building on Heretic by p-e-w.
- Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
- Meta's CyberSecEval; GSM8K (OpenAI); EvalPlus.
- llama.cpp (ggml-org).