library_name: transformers
license: other
license_name: lfm1.0
license_link: LICENSE
pipeline_tag: any-to-any
base_model: LiquidAI/LFM3-Omni-2B
tags:
- lfm3
- lfm3-omni
- abliteration
- heretic
- refusal-removal
- safety-research
- custom_code
- conversational
- audio
- vision
language: - en
- zh
- ar
- fr
- de
- it
- ja
- ko
- pt
- es
- vi
- th
LFM3-Omni-2B-Uncensored-ARA
Refusal-removed derivative of LiquidAI/LFM3-Omni-2B, built with ARA (Arbitrary-Rank Ablation).
GGUF quantisations for llama.cpp (BF16 / Q8_0 / Q4_K_M): LFM3-Omni-2B-Uncensored-ARA.gguf
Research-only derivative. This checkpoint has had its refusal behaviour deliberately removed.
It will comply with requests that the base model would decline, and can produce unsafe,
illegal, or harmful content. It is not intended for production or consumer-facing use.
Read Intended use and limitations before using it.
Produced with Heretic 2.1.0.
Only weight matrices were edited; architecture, tokenizer, processor, chat template and
all other files are byte-identical to the base checkpoint.
| Value | |
|---|---|
| Base model | LiquidAI/LFM3-Omni-2B |
| Modifier | ARA (Arbitrary-Rank Ablation) |
| Refusals (base → this model) | 99/100 → 0/100 |
| KL divergence from base | 0.0470 |
| Weights changed | 12 of 665 tensors |
| Format | safetensors, BF16, 4.04 GB |
| Optimization | 100 Optuna trials, seed 42, Pareto-selected on (refusal count, KL) |
How it was made
Heretic searches for directions in the residual stream that encode refusal, then optimizes
a weight edit that minimises both the number of refusals on a 100-prompt evaluation set and
the KL divergence from the base model's output distribution. ARA performs a full-rank
projection of the refusal direction; the MPOA/"Abliteration" modifier applies a
magnitude-preserving orthogonal ablation. Pareto front trial was exported withheretic-llm==2.1.0 (torch 2.15, transformers 5.22.2, MPS). No training or fine-tuning
was performed.
The upload keeps the base model's original LICENSE, chat template, trust_remote_code
Python modules and processor configs. video_preprocessor_config.json and .gitattributes
were restored from the base repository because save_pretrained() does not emit them.
Quickstart
pip install "transformers>=5.22.2" torch torchvision torchcodec accelerate pillow librosa soundfile
import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "NeosQ/LFM3-Omni-2B-Uncensored-ARA"
device = "cuda" if torch.cuda.is_available() else (
"mps" if torch.backends.mps.is_available() else "cpu"
)
dtype = torch.float32 if device == "cpu" else torch.bfloat16
model = AutoModelForMultimodalLM.from_pretrained(
model_id, dtype=dtype, device_map=device, trust_remote_code=True
).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
conversation = [{"role": "user", "content": "Give me a one-sentence definition of gravity."}]
inputs = processor.apply_chat_template(
conversation,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=512, generation_mode="text")
text = processor.decode(
outputs.sequences[0, inputs["input_ids"].shape[1]:outputs.sequence_lengths[0]],
skip_special_tokens=True,
)
print(text)
trust_remote_code=True is required: this is a custom-code model. Image, video and audio
inputs use the base model's processor.apply_chat_template() interface unchanged; tool
schemas are passed with tools=... and render into the <tools> block of the system turn.
Evaluation
All numbers below are measured against the unmodified base model on the same hardware
and prompt setup. Δ is in percentage points; z is a two-proportion z-test (|z| < 1.96
means the difference is not significant).
Instruction following, commonsense and math
| Benchmark | n | Base | This model | Δ vs base |
|---|---|---|---|---|
| HellaSwag | 200 | 0.485 | 0.475 | -1.0 pp (z=-0.20) |
| PIQA | 200 | 0.635 | 0.630 | -0.5 pp (z=-0.10) |
| WinoGrande | 200 | 0.530 | 0.545 | +1.5 pp (z=+0.30) |
| CommonsenseQA | 200 | 0.350 | 0.345 | -0.5 pp (z=-0.11) |
| SocialIQA | 200 | 0.330 | 0.335 | +0.5 pp (z=+0.11) |
| GSM8K | 200 | 0.710 | 0.685 | -2.5 pp (z=-0.54) |
| MATH-500 | 200 | 0.590 | 0.595 | +0.5 pp (z=+0.10) |
| IFEval | 541 | 0.745 | 0.762 | +1.7 pp (z=+0.64) |
Function calling (BFCL v4, full 1,240 cases)
| Benchmark | n | Base | This model | Δ vs base |
|---|---|---|---|---|
| BFCL v4 simple_python | 400 | 0.922 | 0.930 | +0.8 pp (z=+0.41) |
| BFCL v4 multiple | 200 | 0.965 | 0.955 | -1.0 pp (z=-0.51) |
| BFCL v4 parallel | 200 | 0.620 | 0.785 | +16.5 pp (z=+3.61) |
| BFCL v4 parallel_multiple | 200 | 0.495 | 0.600 | +10.5 pp (z=+2.11) |
| BFCL v4 irrelevance | 240 | 0.733 | 0.637 | -9.6 pp (z=-2.26) |
BFCL is run in the model's native function-calling mode: tool schemas go through the
chat template's tools slot and answers use the native <tool_call> format. One sentence
in the system prompt ("You may call multiple functions in a single response...") is needed
because the template does not say so; without it the model emits a single call per turn andparallel collapses to ~25% for every variant.
Read these numbers together. The differences are driven by how eager the model is to
call a tool, not by function-calling knowledge: in irrelevance (where the correct answer
is to call nothing), this model actually emitted a call in 36.2% of cases. Higherparallel scores therefore come with lower irrelevance scores.
Multimodal (sampled public benchmarks)
| Task | n | Base | This model | Δ |
|---|---|---|---|---|
| POPE (random, yes/no) | 150 | 91.3% | 91.3% | +0.0 pp |
| OmniBench (image+audio, MC) | 120 | 51.7% | 46.7% | -5.0 pp |
| MMMU-Pro vision (MC) | 80 | 20.0% | 20.0% | +0.0 pp |
| WorldSense (audio-visual, MC) | 60 | 45.0% | 43.3% | -1.7 pp |
| VoiceBench BBH (audio→text) | 80 | 62.5% | 58.8% | -3.7 pp |
| FLEURS en_us (ASR, WER↓) | 80 | 4.39 | 4.40 | +0.01 |
| Seed-TTS-eval (TTS→ASR, WER↓) | 30 | 0.93 | 0.93 | +0.00 |
Each task uses the same sampled cases for the base model and this checkpoint. There is no
measurable degradation: POPE is identical case-by-case, ASR WER differs by at most 0.02 pp,
and the remaining gaps are within one standard error. The largest gap is TTS round-trip for
the abliterated variant (+0.66 pp WER on 30 cases, paired-bootstrap CI touching zero).
MMMU-Pro sits at the 20% floor (random is 25%) for every variant and is not informative.
Speech synthesis is scored by round-trip intelligibility (TTS→ASR WER), not by a MOS model.
Video coverage is a 60-case WorldSense sample (audio-visual QA); MVBench / Video-MME,
multilingual ASR and OCR-style document tasks are not covered — see Limitations.
BFCL is extremely sensitive to the harness
| Harness | Base | ARA | Abliteration |
|---|---|---|---|
| Prompt mode (official, pythonic output only) | 45.6% | 59.2% | 33.3% |
| Prompt mode, XML normalised | 78.4% | 74.7% | 77.4% |
| Native FC, bare template | 65.1% | 64.9% | 65.3% |
| Native FC + parallel hint (tables above) | 74.7% | 78.1% | 70.8% |
Prompt-mode scores mostly measure output-format compliance: all three variants pick the
correct function name ~100% of the time, but differ in how often they answer in the
pythonic form the harness expects. Do not compare these values across harnesses.
Long-form generation
Read this if you want an answer longer than a few hundred tokens.
Long output is the weakest part of the base model, not of the ablation. Measured on a
fixed 1,600-token creative-writing prompt, the base checkpoint at its own default decoding
does not write the piece at all: it returns a ~120-character meta paragraph about the topic.
Push it to write at length and it does one of three things — a verbatim loop (the same
100-150-character block repeated 8-11 times), a collapse into a bullet outline, or a drift
into code.
The mechanical cause is text_repetition_window. It defaults to 32 tokens, shorter than one
sentence, so once a paragraph leaves that window nothing penalises it and the model copies it
back word for word. Low temperature makes the attractor stronger, not weaker.
Recommended decoding when the answer is meant to be long:
| Parameter | Base default | Recommended for long output |
|---|---|---|
temperature |
0.8 | 1.0 |
top_p |
0.9 | 0.95 |
repetition_penalty |
1.0 | 1.3 |
text_repetition_window |
32 | 1024 |
max_new_tokens |
- | 1600-3600 |
| Assistant prefill | none | 2-3 opening sentences |
All three prompt-side steps matter:
- Fix the register in the system turn ("output prose only, no outline, no code").
- Enumerate the beats in the user turn and say "do not skip steps".
- Prefill the first two or three sentences as an assistant turn and call
apply_chat_template(..., continue_final_message=True). This single step removes both the
code drift and the outline collapse.
On the fixed prompt this moved 5-gram repetition from 0.30-0.82 down to 0.005-0.065 and took
a one-shot answer from ~600 to ~1,900 characters. The same settings hold for non-fiction: on
a "write a research report" prompt, repetition went from 0.304 to 0.006.
Ceiling. One pass reliably holds 800-2,000 characters. Past that, generate beat by beat
and feed the text so far back as the continued assistant turn instead of asking for the whole
piece at once. Do not use "expand this draft" loops: this model returns the draft unchanged.
On this checkpoint. The ablation makes both failure modes worse at default decoding. A prose request is answered with a Python class (606 characters, repetition 0.301, five lines of class / def); greedy decoding is worse still, 897 characters of which 21 are Chinese. Pushed into a long answer, ARA loops hardest of the three variants — 1,842 characters with one five-sentence block repeated 11 times (repetition 0.816). At the recommended settings it is indistinguishable from the other two (1,953 characters, repetition 0.065), so this is decoding sensitivity rather than lost knowledge: the weights still know how to write, they fall into the attractor sooner. Content-level slips are also easier to see: the scene invents props nobody introduced and sometimes mixes up who is acting. If you need clean long prose, fix the decoding as above; the direction-abliterated checkpoint is a little steadier if you have the choice.
Which variant should I use?
ARA removes refusals completely (0/100) but shifts the output distribution more
(KL 0.0470), is noticeably more eager to call tools, and drops into the long-form failure
modes earlier — see Long-form generation. Abliteration (MPOA)
leaves 3/100 refusals with a much smaller shift (KL 0.0176) and is the steadier of the two on
long output. On knowledge, math and instruction following, both variants are statistically
indistinguishable from the base model; outside long-form decoding, the measurable difference
is in tool-calling eagerness.
Intended use and limitations
Intended: research on refusal mechanisms, abliteration/ARA methodology, safety
evaluation and red-teaming; reproducible benchmark comparisons of ablated checkpoints.
Not intended: production deployments, consumer-facing assistants, anything involving
minors, medical/legal/financial advice, or any setting where safety filtering is expected.
This checkpoint has no refusal guardrail; deploy your own filtering, monitoring and
access controls and comply with local law. By using it you accept full responsibility
for its outputs.
Other limitations:
- HumanEval could not be run (the evaluator's subprocess-per-problem pattern triggers the
sandbox's bulk-delete guard), so coding ability is unmeasured. - Commonsense/math suites use 200 samples each (SE ≈ 3.5 pp); IFEval uses the full 541.
- BFCL covers only the five single-turn non-live categories, not the official overall
leaderboard. - Multimodal coverage is seven sampled public benchmarks (600 cases per checkpoint).
Video is only a 60-case WorldSense sample; MVBench / Video-MME, multilingual ASR,
TTS quality (UTMOS) and OCR-heavy document tasks are not covered, and MMMU-Pro is at
the floor for this model size. - Inference here was serial on MPS. Absolute scores are not comparable to leaderboard
numbers produced with large-batch serving. generation_config.jsonwas re-serialised by transformers 5.22.2; it drops the base
model'stemperature: 0/top_k: 1. Default decoding is greedy (do_sample=false),
so this does not change the results above, but sampling behaviour may differ from base.- Long-form output is the weak point of this model size: at default decoding it loops, drifts
into code or collapses into an outline. See Long-form generation for
the decoding settings that stop it, and for the 800-2,000-character ceiling of a single pass. - This is a community derivative and is not affiliated with or endorsed by Liquid AI.
License and attribution
Distributed under the LFM Open License v1.0, the licence of the base model (seeLICENSE); all its terms, including restrictions on redistribution and use,
apply to this derivative. Base model: LiquidAI/LFM3-Omni-2B.
Reproducing
Method: Heretic 2.1.0, 100 Optuna trials, seed 42, 100-prompt refusal set, KL objective,
Pareto-front selection. Benchmarks: lm-eval 0.4.13 (IFEval, HellaSwag, PIQA, WinoGrande,
CommonsenseQA, SocialIQA, GSM8K, MATH-500) and BFCL v4 (bfcl-eval), both full and
subset runs are archived with the experiment. The experiment report (REPORT.md) and the
summary tables (outputs/final_table.csv, outputs/benchmarks/*.csv) ship with the source
repository; contact the uploader for raw generation logs.