library_name: mlx
license: other
license_name: openmdw-1.1
license_link: https://openmdw.ai/license/1-1/
pipeline_tag: text-generation
base_model: balllvin/Dory1.0-4B-abliterated-Arete-NVFP4
datasets:
- NousResearch/arete-data-pairs
language: - en
tags: - mlx
- nvfp4
- 4-bit
- dory
- abliterated
- arete
Dory 1.0 4B abliterated + Arete, NVFP4 on MLX
The MLX build of balllvin/Dory1.0-4B-abliterated-Arete-NVFP4,
for Apple silicon. It has the same 4-bit codes and block scales, laid out so MLX loads them directly. Use this repo on a
Mac and the NVFP4 repo elsewhere. Results, refusals, training data and method are on the
NVFP4 card; every number there was measured with
vLLM on the NVFP4 file, not on this MLX build.
This model is abliterated and then trained on Nous Research's Arete values data. It is for research. See Limits.
Results (from the NVFP4 build, vLLM, 2 loops, thinking on)
| Build | Harmful refused (99) | Benign refused (100) | MMLU (300) |
|---|---|---|---|
| NVIDIA NVFP4 | 56.6% | 17.0% | 0.830 |
| Abliterated | 13.1% | 0.0% | 0.857 |
| Abliterated + Arete (this model) | 26.3% | 0.0% | 0.833 |
Propaganda Eval fact score, 4 loops: 1.540 (NVIDIA NVFP4 1.823, abliterated 1.993). Arete lowered it: shorter answers
with confident factual errors. Details on the NVFP4 card.
Refusals
The NVFP4 card has the full section: counts per JailbreakBench category and the model's own refusal reasoning. In
short, the refusals come from the model reasoning about harm, law and fairness; the Arete training answers contain
no refusals, and benign prompts are never refused (0 of 100).
Run on a Mac
python3 -m venv venv && . venv/bin/activate
pip install "mlx==0.32.4" \
"git+https://github.com/pmichalas/mlx-lm@208065f3354e8c2645b658493b6aeb94eb0aaa77" \
sentencepiece protobuf
hf download balllvin/Dory1.0-4B-abliterated-Arete-NVFP4-MLX --local-dir dory-arete-mlx
python dory-arete-mlx/tools/serve.py --model dory-arete-mlx --port 8097 --temp 1.0 --top-p 0.95
The plain server needs no tools/ folder:
python -m mlx_lm server --model dory-arete-mlx --port 8097 --temp 1.0 --top-p 0.95
Thinking is on. The model reasons before it answers, in the reasoning field, so a low max_tokens returns emptycontent. Dory is a looped model: 2 loops by default, 4 for hard tasks.
A load test on an Apple silicon Mac (pinned fork, mlx_lm.load) loaded the model in 0.9 s and answered "What is 17
times 23?" with a closed thought and "17 × 23 = 391."
Check or rebuild the conversion
python dory-arete-mlx/tools/convert.py --source <NVFP4 folder> --out my-dory-mlx
python dory-arete-mlx/tools/verify.py --source <NVFP4 folder> --out my-dory-mlx
verify.py re-reads both folders. For this build it found all 507 tensors, with 3,201,564,672 bytes identical to the
NVFP4 build; the 72 gate vectors are the only changed tensors.
How the conversion works
MLX reads NVFP4 codes and block scales through mx.quantized_matmul. It has no input for the tensor scale
(weight_scale_2), so the converter folds it in:
- Same bytes: all 288
weightand 288weight_scaletensors (fused by row and retyped), and 147 bf16 tensors. - Changed: 72 gate vectors (
s_gate,suv_gate), stored in fp32 with their scale folded in. nvfp4_source.safetensorskeeps the source scales and gate vectors;nvfp4_mlx.jsonrecords the source and changes.
Limits
- The model is abliterated. It answers most harmful requests. It is not safe for unsupervised or user-facing use.
- No benchmark was run on this MLX build itself; the numbers above come from the NVFP4 file on vLLM.
- Loading prints a tokenizer warning about
fix_mistral_regex; it also appears with NVIDIA's NVFP4 file.
Credits
- NVIDIA, for Nemotron-Labs Dory 1.0 4B (OpenMDW 1.1).
- OBLITERATUS, for the abliteration method (AGPL-3.0; no OBLITERATUS code is included).
- Nous Research, for the Arete data and the Censorship and Propaganda Eval.
- MLX (Apple), and the pinned mlx-lm fork by pmichalas, which adds Dory support.
Licence
- Weights, config and tokenizer: OpenMDW 1.1, in
LICENSE.OpenMDW-1.1; origin inNOTICE.md. - Code in
tools/: MIT, intools/LICENSE.
This is an independent community build. It is not an NVIDIA or Nous Research product, and neither has reviewed or
endorsed it.