license: other
license_name: lfm1.0
license_link: LICENSE
base_model:
- LiquidAI/LFM3-Omni-2B
- orcarouter/LFM3-1.7B-abliterated
base_model_relation: merge
pipeline_tag: any-to-any
library_name: transformers
tags: - liquid
- lfm3
- lfm3-omni
- omni
- task-arithmetic
- abliterated
- custom_code
LFM3-Omni-2B-abliterated
This is LiquidAI's LFM3-Omni-2B with a change added to its language core. The change is the abliteration of LFM3-1.7B by orcarouter (refusal direction removed).
We kept every audio and vision part of Omni exactly as Liquid released it: the audio encoder and connector, the
vision encoder and connector, the depth decoder that writes speech, and the speech codec. Only the 286 tensors of
the language core changed.
How it was made
Omni's core has the same shape as LFM3-1.7B, but its weights have moved a long way from it. We measured the
distance tensor by tensor: about 54% relative change from LFM3-1.7B and 43% from LFM3-1.7B-Base. So the core
cannot simply be replaced.
We tried a straight swap first. We put the abliterated LFM3-1.7B core in place of Omni's. It broke speech: the word
error rate on speech recognition went from 2.6% to 674.0%, and spoken question
answering fell from 62.0% to 1.0%. The model still heard the words, but it
ignored the task and kept talking.
So we used task arithmetic (Ilharco et al., 2023). We took the difference between orcarouter/LFM3-1.7B-abliterated and LiquidAI/LFM3-1.7B,
tensor by tensor, and added that difference to Omni's core:
new core = Omni core + (changed LFM3-1.7B − LFM3-1.7B)
This moves Omni's core by 1.0% of its size. No training was done on Omni.
Results against the original Omni
All runs used the same fixed eval sets, chosen and hashed before any variant was built, on a Mac with MLX, greedy
decoding (speech output uses the card's sampling: temperature 0.4, repetition penalty 1.2). Each score is shown
with its 95% range. The difference column compares the two models item by item (paired bootstrap). A difference
whose 95% range includes zero is within noise.
| Skill | Original Omni | This model | Difference (95% range) | Reading |
|---|---|---|---|---|
| Speech recognition (LibriSpeech test-clean, 100 clips) | 2.6% (1.9 to 3.4) | 2.7% (2.0 to 3.6) | +0.2 pt (+0.0 to +0.4) | within noise |
| word error rate: the share of words transcribed wrong; lower is better | ||||
| Spoken question answering (VoiceBench OpenBookQA, 100 spoken questions) | 62.0% (52.2 to 70.9) | 63.0% (53.2 to 71.8) | +1.0 pt (+0.0 to +3.0) | within noise |
| accuracy: the share of multiple-choice questions answered right from speech alone; higher is better | ||||
| Image question answering (MMStar, 102 questions, 17 per category) | 64.7% (55.1 to 73.3) | 59.8% (50.1 to 68.8) | -4.9 pt (-9.8 to +0.0) | within noise |
| accuracy: the share of multiple-choice questions about an image answered right; higher is better | ||||
| Speech output (40 sentences, spoken by the model, transcribed back with Parakeet) | 2.3% (0.9 to 3.9) | 2.1% (0.8 to 3.6) | -0.1 pt (-1.0 to +0.5) | within noise |
| word error rate of the transcript of the model's own speech; lower is better |
Text results
Text tasks ran through an OpenAI-compatible server (mlx-vlm) with the same harness for every model. Finance tasks
run with thinking on; the other text tasks with thinking off. The difference column is against the original Omni.
| Task | Original Omni | Abliterated LFM3-1.7B (text only) | This model | Difference vs Omni |
|---|---|---|---|---|
| Function calling, BFCL v3 simple (40) | 90.0% (76.9 to 96.0) | 80.0% (65.2 to 89.5) | 90.0% (76.9 to 96.0) | +0.0 pt (+0.0 to +0.0) |
| share of single tool calls with the right name and arguments | ||||
| Function calling, BFCL v3 live simple (40) | 75.0% (59.8 to 85.8) | 70.0% (54.6 to 81.9) | 77.5% (62.5 to 87.7) | +2.5 pt (+0.0 to +7.5) |
| the same on user-written requests | ||||
| Grade-school math, GSM8K (first 50) | 78.0% (64.8 to 87.2) | 70.0% (56.2 to 80.9) | 82.0% (69.2 to 90.2) | +4.0 pt (-6.0 to +14.0) |
| share of word problems with the right final number | ||||
| Finance agent tasks (20, SEC filings with tools) | 4.2% (0.0 to 9.3) | 11.2% (1.7 to 24.5) | 7.3% (0.6 to 15.6) | +3.1 pt (-5.5 to +13.1) |
| mean numeric score of the submitted answer; 0 when none was submitted | ||||
| Finance agent tasks, answer submitted | 95.0% (76.4 to 99.1) | 30.0% (14.5 to 51.9) | 85.0% (64.0 to 94.8) | -10.0 pt (-25.0 to +0.0) |
| share of tasks where the model called the submit tool |
These sets are small (20 to 50 items), so their 95% ranges are wide. Treat them as a check that nothing broke,
not as a ranking.
Use
examples/infer_mlx.py runs it on a Mac with mlx-vlm 0.7.12 or later. examples/infer_transformers.py runs it with
Transformers 5.20.2 or later on an NVIDIA GPU (bf16) or on CPU or Apple MPS (fp32). Both scripts check text,
speech in, image in and speech out. We ran both on a Mac (Apple M5 Pro): text, speech recognition, an image
description and speech output gave the same answers on both paths, and the spoken sentence transcribed back word for
word. Peak memory was 5.3 GB with MLX (bf16) and 14.4 GB with Transformers on MPS (fp32). The Transformers path was
not run on an NVIDIA GPU. The prompts and generation modes are the same as in Liquid's
LFM3-Omni-2B card.
This is a non-reasoning model, like the original Omni. The chat template, tokenizer, processor and code files are
Liquid's, unchanged.
Limits
- Safety refusals are reduced by the abliteration. You are responsible for how you use it and for what it generates.
- Video input and interleaved speech chat were not measured.
- No 4-bit build is planned.
Changes and licence
Modified file: model.safetensors (language core tensors only). All other files are Liquid AI's originals.graft.json records the method and the sha256 of the inputs and output. The model is released under the
LFM Open License v1.0, which covers derivatives; commercial use is limited to entities under USD 10
million annual revenue (section 5).
Credits
- Liquid AI: LFM3-Omni-2B and LFM3-1.7B.
- orcarouter: LFM3-1.7B-abliterated.
- Task arithmetic: Ilharco et al., Editing Models with Task Arithmetic, ICLR 2023, arXiv:2212.04089.
- Eval data: LibriSpeech (openslr), VoiceBench OpenBookQA (hlt-lab), MMStar (Lin-Chen), BFCL v3, GSM8K. Speech transcribed back with NVIDIA Parakeet TDT 0.6B v3.