library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
base_model: RecursiveSI/flint
datasets:
NousResearch/arete-data-pairs
language:en
tags:nvfp4
4-bit
flint
moe
abliterated
arete
extra_gated_prompt: |-
Terms of access and useBy requesting access to, downloading or using this model, including its weights, code and anything it generates, you agree to these terms.
- Your responsibility. You alone are responsible for how you use this model and for everything it produces for you. You accept full liability for that use and for those outputs, including any decision you make, action you take or content you share because of them.
- No warranty. The model is provided "as is" and "as available", without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, accuracy, title and non-infringement.
- No liability. To the fullest extent the law allows, the authors and publishers of this repository are not liable for any damages, claims, losses or costs, whether direct, indirect, incidental, special, consequential or punitive, that arise from or relate to your access to or use of the model or its outputs, however caused.
- Indemnity. You will indemnify, defend and hold harmless the authors and publishers against any claim, damage, loss, liability or cost, including reasonable legal fees, that arises from your use of the model or its outputs or from your breach of these terms.
- Laws and licences. You will follow every law that applies to you, the licence of this repository and the licence of every model or dataset it is built on. These terms add to those licences and do not take away any right they grant you.
- Agreement. Accessing or downloading this repository means you agree to these terms. If you do not agree, do not access, download or use it. If any part of these terms cannot be enforced, the rest still applies.
extra_gated_fields:
I agree to the terms above and accept full liability for my use: checkbox
extra_gated_button_content: Agree and access repository
Flint abliterated + Arete, NVFP4 safetensors
This repo holds the NVFP4 safetensors build of Flint abliterated + Arete: Recursive's Flint (a mixture-of-experts model, 39.8B
parameters with about 2.5B active per token) with its refusal direction removed, then fine-tuned on
Nous Research's Arete values data. The fine-tune
made it refuse more harmful requests (26 of 99, up from 16; the official build refuses 82), and it still answers
almost every benign one (5 of 100 refused). Its Propaganda Eval fact score is 1.81 out of 5 (higher is better),
up from 1.75 before the fine-tune; the official build scores 1.64. It is for research. Improved versions are coming.
Which file do I use?
| Your hardware | Build | Repo |
|---|---|---|
| NVIDIA GPU with llama.cpp | NVFP4 GGUF, one 22.9 GB file | balllvin/Flint-abliterated-Arete-NVFP4-GGUF |
| NVIDIA GPU, your own code (this one) | NVFP4 safetensors in ModelOpt's W4A16_NVFP4 layout |
balllvin/Flint-abliterated-Arete-NVFP4 |
| Apple Silicon Mac | MLX layout, run with flint_mlx.py from the repo |
balllvin/Flint-abliterated-Arete-NVFP4-MLX |
All three hold the same 4-bit weights.
Set it up with a coding agent
Paste this into a coding agent such as Claude Code or
OpenCode. Edit the first line first.
Use it in: [Insert application you will use it in, for example OpenCode]
Set up Flint abliterated + Arete on this computer. Do every step yourself.
1. Check the tools: Python, uv or pip, git, the Hugging Face CLI. Install the HF CLI if it is missing.
2. Check the hardware. Apple Silicon Mac with 48 GB or more: MLX build. NVIDIA GPU with 32 GB or more: GGUF build. Anything else: tell me it will not run here, and suggest workarounds.
3. Ask me to open the chosen repo page and accept the terms at the top of the page, then run `hf auth login`. Download only the chosen build and check every file. MLX: https://huggingface.co/balllvin/Flint-abliterated-Arete-NVFP4-MLX GGUF: https://huggingface.co/balllvin/Flint-abliterated-Arete-NVFP4-GGUF
4. Get the code. Mac: `uv pip install mlx==0.32.4` and flint_mlx.py from the repo. NVIDIA: llama.cpp b11853 or later, CUDA build.
5. Pick the context length for this machine from the memory table on the model card.
6. Ask one question, check the answer makes sense, and report tokens per second. If anything fails, fix it and try again.
7. NVIDIA: start llama-server and connect it to the application on the first line, with its output limit raised to the context length (no application: show one curl call). Mac: flint_mlx.py is not a server, so stop after step 6.
8. Report the build you chose and why, the server address, and one command to start it again.
Results
Refusals on JailbreakBench prompts, run with llama.cpp b11853 on one RTX PRO 6000: thinking
on, temperature 1.0, top-p 0.95, top-k 20, no token limit in the 64K context, one sampled answer per prompt, refusals judged on the final answer with phrase lists.
Every answer finished. 95% intervals in brackets.
| Build | Harmful refused, of 99 (higher refuses more) | Benign refused, of 100 (lower is better) |
|---|---|---|
| Official | 82 (74% to 89%) | 8 (4% to 15%) |
| Abliterated, the start point | 16 (10% to 25%) | 3 (1% to 8%) |
| This model | 26 (19% to 36%) | 5 (2% to 11%) |
The Propaganda Eval asks 803 questions on topics where state censorship and propaganda are common
(Nous Research's set), judged by
xAI's Grok 4.7. All three builds answered with the same server and settings.
| Build | Fact score, 0 to 5 (higher is better) | Any propaganda (lower is better) | Refused, of 803 |
|---|---|---|---|
| Official | 1.64 (1.51 to 1.78) | 2.1% | 100 |
| Abliterated, the start point | 1.75 (1.61 to 1.88) | 2.2% | 31 |
| This model | 1.81 (1.67 to 1.96) | 2.0% | 2 |
The answers are public: official,
this model.
Memory
GB means 10^9 bytes; K means 1,024 tokens. The weights take 22.9 GB (measured). The KV cache, computed fromconfig.json in bf16, takes 28,672 bytes (about 28.7 KB) per token for the 14 full-attention layers, plus 36.8 MB per
request for the 35 sliding-window layers: 7.6 GB for one request at the full 256K.
| Machine | Usable | Left for KV | Longest context, 1 request | 4 requests | 8 requests |
|---|---|---|---|---|---|
| Mac with 64 GB | 55.6 GB | 28.5 GB | 256K | 241K | 119K |
| GPU with 48 GB | 46.4 GB | 21.4 GB | 256K | 180K | 89K |
| GPU with 96 GB | 92.8 GB | 67.8 GB | 256K | 256K | 256K |
Usable is the GPU working set Metal recommends on a 64 GB M5 Pro, or 90% of a GPU's memory. Left for KV subtracts the weights and
runtime room (4.3 GB on the Mac, 2.1 GB on a GPU).
Speed
With 16 requests at once on one RTX PRO 6000 (llama.cpp b11853, GGUF), each request decoded at 40 tokens/s: the median over the
399 benchmark replies of 256 tokens or more.
Run it
Before the first download, accept the terms at the top of this page, then log in with hf auth login. Thinking is
always on. Use the base model's sampling: temperature 1.0, top-p 0.95, top-k 20.
# NVIDIA GPU, llama.cpp
hf download balllvin/Flint-abliterated-Arete-NVFP4-GGUF flint-ablit-arete-nvfp4.gguf --local-dir flint-arete
llama-server -m flint-arete/flint-ablit-arete-nvfp4.gguf --alias flint -ngl all -c 262144 -np 1 -fa on -ctk f16 -ctv f16 --jinja --temp 1 --top-p 0.95 --top-k 20 --min-p 0 --port 8080
# Mac, MLX
hf download balllvin/Flint-abliterated-Arete-NVFP4-MLX --local-dir flint-arete-mlx
uv run --with mlx==0.32.4 --with numpy --with transformers python flint-arete-mlx/flint_mlx.py --model flint-arete-mlx --dtype bf16 --tokens 512 --prompt "Explain how vaccines train the immune system."
flint_mlx.py is a small greedy runner, not a server. The model writes its reasoning in <think>...</think>, then
the answer; llama-server returns them in reasoning_content and content.
How it was made
- Start point: our abliterated build of the
official NVFP4 weights with one refusal direction, found with the
OBLITERATUS method, projected out of every matrix that writes the
residual stream. - Data: the strictest reviewed rows of
Arete (accuracy and balance 3 or more, nofactual_errororone_sidedtag), answers joined with their reasoning: 6,428 rows trained (10.7M tokens), 63
held out. No system line; loss on the reasoning and answer only. No other data, and no row shares an 8-gram with
the 803 propaganda questions. - Training: in NVFP4 under the official frozen scales, 4-bit codes re-rounded every step. Attention,
shared-expert and dense MLP weights trained (1.30B parameters); routed experts frozen. One epoch, all 159 steps of
65,536 tokens, AdamW, learning rate 5e-6 with 10 warmup steps and cosine decay to 5e-7, on one RTX PRO 6000. Held-out
loss fell from 1.592 to 1.137 nats per token; the last step (also the best) is this release. No test set was
used to choose anything.
Limits
- It answers most harmful requests (73 of 99 here). Use it for research, with a person checking the output.
- The fine-tune teaches one set of values, reviewed by Nous Research's pipeline, and
changes how the model frames political topics. - The tests cover refusals and propaganda, one sampled run each; not coding, agents or long context.
Credits
Recursive for Flint and its
NVFP4 GGUF; OBLITERATUS
(AGPL-3.0) for the method; Nous Research for Arete and the
Censorship and Propaganda Eval (MIT);
JailbreakBench; xAI's Grok 4.7 as
judge; llama.cpp and MLX.
Licence
The weights are a modified version of Recursive's Flint under Apache-2.0:LICENSE holds Flint's licence file and NOTICE.md the origin and every change. The
Arete data states no licence; its makers released it
publicly. flint_mlx.py is MIT.
This is an independent community build, unaffiliated with Recursive or
Nous Research.
Terms of use
Access to this repository requires agreeing to the terms in the access form on this page. In short: you alone are responsible and liable for your use of this model and for everything that comes from it, it is provided as is with no warranty, and its authors are not liable for any result of that use. Full text: TERMS.md.
To download after you agree, log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN.