library_name: onnxruntime
license: other
license_name: lfm1.0
license_link: LICENSE
pipeline_tag: image-text-to-text
base_model: balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4-MLX
base_model_relation: quantized
tags:
- lfm3
- onnx
- webgpu
- onnxruntime-web
LFM3-Omni-2B-Abliterated-Arete-ONNX-WebGPU
balllvin/LFM3-Omni-2B-Abliterated-Arete-NVFP4-MLX as ONNX for the browser (WebGPU,
onnxruntime-web). Chat with it in the browser: LFM3-WebGPU.
It answers without a thinking block, as the source release does. Text and images in, text out.
Parity with the bf16 source
| Check | Result |
|---|---|
| Same top token as bf16, all positions | 94.5% |
| Same top token, generated part | 97.5% |
| Mean KL from bf16 (nats per token) | 0.0327 |
| Greedy tokens equal to bf16 before the first difference | 35 of 209 |
| Reference for the same check on LiquidAI's own q4f16 vs its bf16 | 93.1% top token, KL 0.0503 |
Six fixed chat prompts, 48 greedy tokens each from transformers bf16, teacher forced through the ONNX graph with
the caches carried across prefill chunks of 32 tokens (ONNX Runtime CPU).
Speed
256 new tokens, the same prompt, median of 3 runs.
| Where | Decode tok/s | Prefill tok/s |
|---|---|---|
| Apple M5 Pro, headless Chrome | 15.0 | 227 |
How it was built
- Source weights: the bf16 task vector graft rebuilt with
brainspa graft build --method task_vector(host LiquidAI/LFM3-Omni-2B, donor balllvin/lfm3-1.7b-arete-ckpt ar1/best, base LiquidAI/LFM3-1.7B); its sha256 230c9167baead0e521450939961bef070f7e9a1f64ceaac0e5d9f530c29d4bb3 equals themodel.safetensorssha256 in the release's graft.json. - Graph: LiquidAI's ONNX Runtime GenAI graph, LiquidAI/LFM3-Omni-2B-ONNX@d2e656f0bddfaa2b8ebd4bfbe9d88de7897ee592 onnx/decoder_q4f16.onnx, unchanged. Only the weights differ.
- Quantization: the same recipe as that graph: ONNX Runtime k-quant, asymmetric, 32 weights per scale, weights
cast to fp16 first; the embedding, the LM head and the same projections as LiquidAI's file stay in int8, the rest
are int4. Run on LiquidAI's own bf16 source, this code gives back 379 of 594
initializers byte for byte; the other 215 differ in a few bytes, 0.0032% of all bytes (float rounding in the scale search). - Vision encoder: LiquidAI/LFM3-Omni-2B-ONNX@d2e656f0bddfaa2b8ebd4bfbe9d88de7897ee592 onnx/vision_encoder_q8.onnx, byte for byte. The Omni grafts change only the language model, so vision weights are the host model's.
- Parity is measured on the language core alone (
brainspa graft extract), which is what the decoder runs for text. - Speed was measured in headless Chrome while the Mac was busy with other jobs (load average 10 to 12), so it is a low bound.
Files
| File | Size |
|---|---|
onnx/decoder_q4f16.onnx |
0.3 MB |
onnx/decoder_q4f16.onnx_data |
1.41 GB |
onnx/vision_encoder_q8.onnx |
0.1 MB |
onnx/vision_encoder_q8.onnx_data |
0.20 GB |
Total download 1.61 GB. Use import * as ort from 'onnxruntime-web/webgpu' and prefill in chunks of at most
1024 tokens. Inputs and outputs are those of the LiquidAI graph.
Licence
LFM Open License v1.0 (see LICENSE) and NOTICE.md, as the source release.