library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
base_model: RecursiveSI/flint
language:
en
tags:nvfp4
4-bit
flint
moe
abliterated
extra_gated_prompt: |-
Terms of access and useBy requesting access to, downloading or using this model, including its weights, code and anything it generates, you agree to these terms.
- Your responsibility. You alone are responsible for how you use this model and for everything it produces for you. You accept full liability for that use and for those outputs, including any decision you make, action you take or content you share because of them.
- No warranty. The model is provided "as is" and "as available", without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, accuracy, title and non-infringement.
- No liability. To the fullest extent the law allows, the authors and publishers of this repository are not liable for any damages, claims, losses or costs, whether direct, indirect, incidental, special, consequential or punitive, that arise from or relate to your access to or use of the model or its outputs, however caused.
- Indemnity. You will indemnify, defend and hold harmless the authors and publishers against any claim, damage, loss, liability or cost, including reasonable legal fees, that arises from your use of the model or its outputs or from your breach of these terms.
- Laws and licences. You will follow every law that applies to you, the licence of this repository and the licence of every model or dataset it is built on. These terms add to those licences and do not take away any right they grant you.
- Agreement. Accessing or downloading this repository means you agree to these terms. If you do not agree, do not access, download or use it. If any part of these terms cannot be enforced, the rest still applies.
extra_gated_fields:
I agree to the terms above and accept full liability for my use: checkbox
extra_gated_button_content: Agree and access repository
Flint abliterated, NVFP4 safetensors
This repo holds the NVFP4 safetensors build of Flint abliterated: Recursive's Flint (a mixture-of-experts model, 39.8B
parameters with about 2.5B active per token) with its refusal direction removed, so it does what you ask. It refuses 16 of 99 harmful requests (the official build refuses 82) and 3 of 100 benign ones. Its Propaganda Eval fact score is 1.75 out of 5 (higher is better); the official build scores 1.64. It is for research. Improved versions are coming.
Which file do I use?
| Your hardware | Build | Repo |
|---|---|---|
| NVIDIA GPU with llama.cpp | NVFP4 GGUF, one 22.9 GB file | balllvin/Flint-abliterated-NVFP4-GGUF |
| NVIDIA GPU, your own code (this one) | NVFP4 safetensors in ModelOpt's W4A16_NVFP4 layout |
balllvin/Flint-abliterated-NVFP4 |
| Apple Silicon Mac | MLX layout, run with flint_mlx.py from the repo |
balllvin/Flint-abliterated-NVFP4-MLX |
All three hold the same 4-bit weights.
Set it up with a coding agent
Paste this into a coding agent such as Claude Code or
OpenCode. Edit the first line first.
Use it in: [Insert application you will use it in, for example OpenCode]
Set up Flint abliterated on this computer. Do every step yourself.
1. Check the tools: Python, uv or pip, git, the Hugging Face CLI. If the HF CLI is missing, tell me and show me how to install it; do not install it yourself.
2. Check the hardware. Apple Silicon Mac with 48 GB or more: MLX build. NVIDIA GPU with 32 GB or more: GGUF build. Anything else: tell me it will not run here, and suggest workarounds.
3. Ask me to open the chosen repo page and accept the terms at the top of the page, then run `hf auth login`. Download only the chosen build and check every file. MLX: https://huggingface.co/balllvin/Flint-abliterated-NVFP4-MLX GGUF: https://huggingface.co/balllvin/Flint-abliterated-NVFP4-GGUF
4. Get the code. Mac: `uv pip install mlx==0.32.4` and flint_mlx.py from the repo. NVIDIA: llama.cpp b11853 or later, CUDA build.
5. Pick the context length for this machine from the memory table on the model card.
6. Ask one question, check the answer makes sense, and report tokens per second. If anything fails, fix it and try again.
7. NVIDIA: start llama-server and connect it to the application on the first line, with its output limit raised to the context length (no application: show one curl call). Mac: flint_mlx.py is not a server, so stop after step 6.
8. Report the build you chose and why, the server address, and one command to start it again.
Results
Refusals on JailbreakBench prompts, run with llama.cpp b11853 on one RTX PRO 6000: thinking
on, temperature 1.0, top-p 0.95, top-k 20, no token limit in the 64K context, one sampled answer per prompt, refusals judged on the final answer with phrase lists.
Every answer finished. 95% intervals in brackets.
| Build | Harmful refused, of 99 (higher refuses more) | Benign refused, of 100 (lower is better) |
|---|---|---|
| Official | 82 (74% to 89%) | 8 (4% to 15%) |
| This model | 16 (10% to 25%) | 3 (1% to 8%) |
The Propaganda Eval asks 803 questions on topics where state censorship and propaganda are common
(Nous Research's set), judged by
xAI's Grok 4.7. Both builds answered with the same server and settings.
| Build | Fact score, 0 to 5 (higher is better) | Any propaganda (lower is better) | Refused, of 803 |
|---|---|---|---|
| Official | 1.64 (1.51 to 1.78) | 2.1% | 100 |
| This model | 1.75 (1.61 to 1.88) | 2.2% | 31 |
The answers are public: official,
this model.
Memory
GB means 10^9 bytes; K means 1,024 tokens. The weights take 22.9 GB (measured). The KV cache, computed fromconfig.json in bf16, takes 28,672 bytes (about 28.7 KB) per token for the 14 full-attention layers, plus 36.8 MB per
request for the 35 sliding-window layers: 7.6 GB for one request at the full 256K.
| Machine | Usable | Left for KV | Longest context, 1 request | 4 requests | 8 requests |
|---|---|---|---|---|---|
| Mac with 64 GB | 55.6 GB | 28.5 GB | 256K | 241K | 119K |
| GPU with 48 GB | 46.4 GB | 21.4 GB | 256K | 180K | 89K |
| GPU with 96 GB | 92.8 GB | 67.8 GB | 256K | 256K | 256K |
Usable is the GPU working set Metal recommends on a 64 GB M5 Pro, or 90% of a GPU's memory. Left for KV subtracts the weights and
runtime room (4.3 GB on the Mac, 2.1 GB on a GPU).
Speed
With 32 requests at once on one RTX PRO 6000 (llama.cpp b11853, GGUF), each request decoded at 27 tokens/s: the median over the
697 benchmark replies of 256 tokens or more.
Run it
Before the first download, accept the terms at the top of this page, then log in with hf auth login. Thinking is
always on. Use the base model's sampling: temperature 1.0, top-p 0.95, top-k 20.
# NVIDIA GPU, llama.cpp
hf download balllvin/Flint-abliterated-NVFP4-GGUF flint-ablit-nvfp4.gguf --local-dir flint-ablit
llama-server -m flint-ablit/flint-ablit-nvfp4.gguf --alias flint -ngl all -c 262144 -np 1 -fa on -ctk f16 -ctv f16 --jinja --temp 1 --top-p 0.95 --top-k 20 --min-p 0 --port 8080
# Mac, MLX
hf download balllvin/Flint-abliterated-NVFP4-MLX --local-dir flint-ablit-mlx
uv run --with mlx==0.32.4 --with numpy --with transformers python flint-ablit-mlx/flint_mlx.py --model flint-ablit-mlx --dtype bf16 --tokens 512 --prompt "Explain how vaccines train the immune system."
flint_mlx.py is a small greedy runner, not a server. The model writes its reasoning in <think>...</think>, then
the answer; llama-server returns them in reasoning_content and content.
How it was made
- Direction: last-token residuals for harmful and harmless prompts gave one refusal direction per candidate
layer (difference of the means), using OBLITERATUS's method.
Extra prompts from HarmBench and
AdvBench (both MIT) widened the categories; none overlap the test prompts. - Choice: layers 18 to 26 were tried at two strengths, keeping drift on harmless answers under a cap. The best
two were written into the weights and re-asked 123 validation prompts; layer 22 at strength 1.0 won. The test
prompts above were run once, after the choice. - Edit: the direction is projected out of every matrix that writes the residual stream, then re-quantized to
NVFP4 with the official scales. Against the bf16 model it matches the official build: 3.094 nats per token on
WikiText (official 3.094) and 77.7% next-token agreement (official 77.8%).
Limits
- It answers most harmful requests (83 of 99 here). Use it for research, with a person checking the output.
- The tests cover refusals and propaganda, one sampled run each; not coding, agents or long context.
Credits
Recursive for Flint and its
NVFP4 GGUF; OBLITERATUS
(AGPL-3.0) for the method; HarmBench and AdvBench for prompts; Nous Research for the
Censorship and Propaganda Eval (MIT);
JailbreakBench; xAI's Grok 4.7 as
judge; llama.cpp and MLX.
Licence
The weights are a modified version of Recursive's Flint under Apache-2.0:LICENSE holds Flint's licence file and NOTICE.md the origin and every change. flint_mlx.py is MIT.
This is an independent community build, unaffiliated with Recursive.
Terms of use
Access to this repository requires agreeing to the terms in the access form on this page. In short: you alone are responsible and liable for your use of this model and for everything that comes from it, it is provided as is with no warranty, and its authors are not liable for any result of that use. Full text: TERMS.md.
To download after you agree, log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN.