library_name: transformers
license: other
license_name: lfm1.0
license_link: https://www.liquid.ai/lfm-license
pipeline_tag: any-to-any
base_model:
LiquidAI/LFM3-Omni-2B
balllvin/LFM3-1.7B-Abliterated-Arete-NVFP4
base_model_relation: merge
datasets:NousResearch/arete-data-pairs
language:en
tags:transformers
nvfp4
4-bit
lfm3
lfm3-omni
abliterated
arete
extra_gated_prompt: |-
Terms of access and useBy requesting access to, downloading or using this model, including its weights, code and anything it generates, you agree to these terms.
- Your responsibility. You alone are responsible for how you use this model and for everything it produces for you. You accept full liability for that use and for those outputs, including any decision you make, action you take or content you share because of them.
- No warranty. The model is provided "as is" and "as available", without warranties of any kind, express or implied, including merchantability, fitness for a particular purpose, accuracy, title and non-infringement.
- No liability. To the fullest extent the law allows, the authors and publishers of this repository are not liable for any damages, claims, losses or costs, whether direct, indirect, incidental, special, consequential or punitive, that arise from or relate to your access to or use of the model or its outputs, however caused.
- Indemnity. You will indemnify, defend and hold harmless the authors and publishers against any claim, damage, loss, liability or cost, including reasonable legal fees, that arises from your use of the model or its outputs or from your breach of these terms.
- Laws and licences. You will follow every law that applies to you, the licence of this repository and the licence of every model or dataset it is built on. These terms add to those licences and do not take away any right they grant you.
- Agreement. Accessing or downloading this repository means you agree to these terms. If you do not agree, do not access, download or use it. If any part of these terms cannot be enforced, the rest still applies.
extra_gated_fields:
I agree to the terms above and accept full liability for my use: checkbox
extra_gated_button_content: Agree and access repository
LFM3-Omni-2B abliterated + Arete, NVFP4 for NVIDIA GPUs
Liquid AI's LFM3-Omni-2B, which hears, sees, reads and speaks, with the changes of our LFM3-1.7B abliterated + Arete (trained on Nous Research's Arete data) added to its language core, in 4-bit NVFP4 weights. It heard and read images and video within noise of the original Omni in our tests, and its speech output was slightly less clear. It is for research, and improved versions are coming.
This repo holds the NVFP4 build (ModelOpt W4A16_NVFP4 safetensors) for NVIDIA GPUs. The NVFP4-MLX twin holds the same 4-bit weights repacked for Apple Silicon Macs.
Which file do I use?
| Your hardware | Build | Repo |
|---|---|---|
| NVIDIA GPU (this one) | NVFP4, for vLLM on Blackwell GPUs | balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4 |
| Apple Silicon Mac | NVFP4-MLX, for MLX | balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4-MLX |
Both builds hold the same 4-bit weights. The MLX build was repacked from the model.safetensors in this repo: its nvfp4_build.json records that file's sha256 (0e9e049f...).
Set it up with a coding agent
Paste this into a coding agent such as Claude Code or OpenCode, with your app in the brackets.
Download LFM3-Omni-2B abliterated + Arete and set it up for [your app, for example OpenCode]. Do every step yourself, except the account steps, which are mine.
1. Check whether the Hugging Face CLI (hf) is installed. If it isn't, tell me and show me how to install it. Don't install it yourself.
2. Check this computer. NVIDIA Blackwell GPU (RTX 50xx, RTX PRO 6000, B200): NVIDIA build. Apple Silicon Mac: Mac build at https://huggingface.co/balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4-MLX. Anything else: tell me it won't run here and suggest workarounds (for example the bf16 build at https://huggingface.co/balllvin/LFM3-Omni-2B-abliterated with Transformers).
3. Walk me through the account steps: make a Hugging Face account if I don't have one, accept the terms on the model page, and make a token. Then run hf auth login for me, or tell me how.
4. Download only the build for this computer and check every file. NVIDIA: https://huggingface.co/balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4 Mac: https://huggingface.co/balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4-MLX
5. Get the code. NVIDIA: install a vLLM or SGLang build that lists Lfm3OmniForConditionalGeneration and supports ModelOpt W4A16_NVFP4; tell me if your build lists neither. Mac: pip install "mlx-vlm==0.7.12" "mlx-audio==0.6.4".
6. Pick the context length from the memory section on the model card.
7. Ask it one question, check the answer makes sense, and tell me the tokens per second. Then test speech and images, for example with examples/infer_mlx.py from the Mac model repo on the same audio and photo, and play back the speech it writes.
8. Start the OpenAI compatible server and connect it to my app, with the app's output limit raised to the context length. No app: show me one curl call.
9. Tell me which build you chose, the server address if there is one, and one command to run it again.
Results
The results below are for the same weights measured in the NVFP4-MLX twin and the original Omni (bf16) on speech, images and video, on a Mac with mlx-vlm, greedy decoding, no token limit, one run each. We did not re-run them on a GPU. Omni is a non-reasoning model (Liquid AI's card: thinking mode is not supported), so it answers without thinking.
| Test | Original Omni | This model | Difference (95% CI) |
|---|---|---|---|
| Speech recognition, 100 LibriSpeech clips (word error rate, lower is better) | 2.58% | 2.84% | +0.26 points (−0.05 to +0.62) |
| Spoken questions, 100 VoiceBench OpenBookQA items | 62.0% | 62.0% | 0.0 points (−4.0 to +4.0) |
| Image questions, 102 MMStar items | 65.7% | 61.8% | −3.9 points (−9.8 to +2.0) |
| Video questions, 50 TempCompass clips | 68.0% | 68.0% | 0.0 points (−8.0 to +8.0) |
| Speech output, 40 sentences read back by Parakeet (word error rate) | 2.27% | 3.07% | +0.80 points (+0.24 to +1.60) |
Differences are paired over the same items. Speech output is the one measurable change: its word error rate rose by 0.8 points. Two image answers ran on until they filled the context, and count as wrong. On text, this build called the right function with the right arguments on 92.5% of 40 simple and 70% of 40 live simple BFCL v3 requests (original Omni: 90% and 75%), and solved 74% of the first 50 GSM8K math problems (original: 76%).
Memory
The weights take 2.7 GB (2.5 GB for the model, 0.14 GB for the speech codec). The KV cache (bf16) grows by 12 KB per token in each request, plus a fixed 2.8 MB per request; audio, images and video enter the context as tokens. The longest context is 128,000 tokens, which needs 1.5 GB of KV cache per request. These numbers are computed from config.json.
Run it
Accept the terms at the top of this page, then log in with hf auth login (or set HF_TOKEN).
Serving this repo needs a runtime that lists Lfm3OmniForConditionalGeneration and supports ModelOpt W4A16_NVFP4. Check that your vLLM or SGLang build does: the vLLM wheel pinned in our LFM3-1.7B card predates LFM3-Omni support, so we did not run this build on a GPU.
# In an environment with a vLLM build that supports LFM3-Omni and NVFP4:
hf download balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4 --local-dir lfm3-omni
vllm serve lfm3-omni --served-model-name lfm3-omni --max-model-len 131072
For text and images, that serves the OpenAI chat API. The Mac twin also runs speech and images end to end with examples/infer_mlx.py.
How it was made
We added the difference between our bf16 LFM3-1.7B abliterated + Arete and LiquidAI/LFM3-1.7B to the Omni language core (task arithmetic), which moves the core by 1.3%. Then we rounded the core's 72 feed-forward weight matrices to NVFP4 and kept everything else, including the audio, vision and speech parts, as Liquid AI released them. We did no training on Omni.
This repo is the ModelOpt NVFP4 source of the MLX twin. The MLX folder copies its 4-bit codes and block scales byte for byte and folds the tensor scales into norms for MLX; this folder keeps the tensor scales as weight_scale_2 and the ModelOpt hf_quant_config.json. No re-quantization was needed to publish this build.
Limits
- The language core comes from an abliterated model, so it answers most harmful requests. Use it for research, with a person checking its output.
- Serving depends on runtime support for LFM3-Omni with NVFP4; this GPU build was not served in our tests.
- Each number is one run on one setup.
Licence
LFM Open License v1.0, from the base models (LICENSE; NOTICE.md lists our changes). Commercial use is licensed only below 10 million US dollars of annual revenue. This is an independent build, not affiliated with Liquid AI, OrcaRouter or Nous Research.
Terms of use
Access to this repository requires agreeing to the terms in the access form on this page. In short: you alone are responsible and liable for your use of this model and for everything that comes from it, it is provided as is with no warranty, and its authors are not liable for any result of that use. Full text: TERMS.md.
To download after you agree, log in with the Hugging Face CLI (hf auth login) or set HF_TOKEN.