library_name: mlx
license: apache-2.0
pipeline_tag: text-generation
base_model: sensenova/SenseNova-Loop-9B
base_model_revision: 329a65f701ba8c48b73c30d0cbaa6f0449733673
datasets:
- NousResearch/arete-data-pairs
- HuggingFaceFW/fineweb-edu
tags: - mlx
- 4-bit
- nvfp4
- abliterated
- arete
- sense-nova-loop
SenseNova-Loop-9B abliterated + Arete, NVFP4 -> MLX 4-bit
The Apple Silicon build of
balllvin/SenseNova-Loop-9B-abliterated-arete-NVFP4:
the final NVFP4 build converted to MLX affine 4-bit (7.4 GB), with the looped 56-of-32 layer plan and the
mHC routers ported to MLX. It is the final MLX build of this family. It is for research.
Builds
| Build | Repo |
|---|---|
| QAD base (NVFP4) | balllvin/SenseNova-Loop-9B-NVFP4 |
| abliterated (NVFP4) | balllvin/SenseNova-Loop-9B-abliterated-NVFP4 |
| + Arete (NVFP4) | balllvin/SenseNova-Loop-9B-abliterated-arete-NVFP4 |
| QAD base, MLX 4-bit | balllvin/SenseNova-Loop-9B-NVFP4-MLX |
| + Arete, MLX 4-bit | this repo |
Speed and memory
M5 Pro, 64 GB, 48-64 tokens out, load 7-15 from other work, so conservative:
| Mode | Steady footprint | Decode tok/s | End-to-end, 64 tok | TTFT |
|---|---|---|---|---|
| default, 8-bit head | 7.5 GB | 20.8 | 18.0 | 0.44 s |
--fast-head |
~8.0 GB | 26.6 | 22.8 | 0.25-0.32 s |
The original MLX build of these weights measured 8.1 tok/s. With --fast-head and --max-batch 8,
server-path concurrency is 22.6 / 33.2 / 53.9 / 51.3 total tok/s at 1 / 2 / 4 / 8 streams (per stream 22.6 /
16.6 / 13.5 / 6.4-11.3). The server is on demand: it starts with the model, warms up the common shapes, and
exits after 30 minutes idle.
Local MLX gate
On 160 teacher-forced WikiText positions:
- Router + bf16 trunk: NLL 3.04790 against 3.04762 for the original iterative fp32 path (+0.00028),
top-1 98.1%. - Default 8-bit head: NLL 3.04775 against 3.04790 for bf16 (-0.00015), top-1 99.4%.
- Fast head: NLL -0.00079 against bf16, top-1 199/199, mean KL 0.000544.
The sealed refusal/MMLU/NLL results are on the NVFP4 card;
they were not re-run on this MLX conversion.
Run it
python tools/serve.py --model . --port 8099 --kv-bits 8 --max-batch 4 --loops 2 --head-bits 8
# faster, +0.57 GB
python tools/serve.py --model . --port 8099 --kv-bits 8 --max-batch 8 --loops 2 --fast-head
The server renders the full message list, supports SSE streaming, and exits after 30 minutes idle
(--idle-exit 0 keeps it up). --loops 1 is faster (27.3 ms/token) but WikiText NLL is 3.921 against 3.041
at two loops; 2 is the trained setting.
How it was made
- Conversion: the Arete NVFP4 build to MLX affine 4-bit (group 64), 7.4 GB.
- Port: the looped execution plan (56 visits of 32 physical layers) and the mHC routers; the gates stay
fp32 but return bf16, so the trunk runs bf16. - Kernels: the checkpoint's 20 alternating 2x2 normalisations are a Mobius map; their exact k-step
closed form is compiled for the decode shape, and the LM head is 8-bit by default (fast head optional). - Server: on demand, with warmup, streaming and loop selection.
Limits
- The sealed refusal/MMLU/NLL table is the NVFP4 source build; it was not re-run on the MLX conversion. The
local gate above is 160 WikiText positions. - Abliteration is behavioural; it answers 93 of 99 harmful prompts in the sealed set. It is not
safety-aligned. --loops 1is not the trained model.- Speed and memory are one M5 Pro; the load was 7-15 from other work, so the numbers are conservative.
Credits
balllvin/SenseNova-Loop-9B-abliterated-arete-NVFP4
for the source build; sensenova/SenseNova-Loop-9B for
the base; NousResearch/arete-data-pairs for
Arete; FineWeb-Edu for the QAD corpus;
MLX and mlx-lm for the runtime;
JailbreakBench and
AdvBench for the sealed prompts on the NVFP4 card.
Licence
The weights are a modified version of SenseNova-Loop-9B
under Apache-2.0. The Arete data states no licence; its makers
released it publicly. This is an independent community build, unaffiliated with the model's authors.