license: other
license_name: lfm-open-license-v1.0
license_link: LICENSE
base_model: AtomicChat/LFM3-1.7B-Uncensored-GGUF
library_name: mlx
pipeline_tag: text-generation
language:

• en
• zh
• de
  tags:
• mlx
• apple-silicon
• liquid
• lfm3
• conversational
• tool-calling
• custom_code
• uncensored
• abliterated
• dequantized
• float32
• not-for-all-audiences

LFM3-1.7B Uncensored · MLX · dequantized float32

An MLX port of AtomicChat/LFM3-1.7B-Uncensored-GGUF
for Apple silicon. AtomicChat publishes this model only as quantized GGUF files, so the weights here are its
AD-Q6_K file dequantized to float32. They are the exact float32 values of the quantized weights, not the weights
from before quantization. Dequantizing adds no precision and cannot undo quantization error.

The port is 4.4 times the size of the GGUF, and it decodes 4 to 5 times slower than the GGUF does under llama.cpp
(speed). Use this port if you need the model in MLX. Otherwise, the GGUF is smaller and faster.

AtomicChat removed the model's safety refusals. It applies no safety filtering of its own.

Is it identical to the source?

Greedy output matched in every test, and the weights match bit for bit. The logits are close but not identical.

llama.cpp's defaults do not give one fixed set of logits to match. Its batched Metal path computes in half precision,
so its logits change with batch size. The float baseline uses batches small enough to avoid that path. The port meets
the limits against that baseline, and two checks against the batched path, written before this was known, stay reported
as misses (evaluation).

These results cover the tested inputs on one machine. They are not proof of identical behavior on every input.

Model Details

AD-Q6_K is the highest-fidelity rung AtomicChat publishes. Of its 152 models, none is an unquantized copy of these
weights. The file holds 133 F32, 66 Q8_0 and 88 Q6_K tensors, and the F32 tensors are copied bit for bit.

How to Get Started

mlx-lm has no built-in LFM3 architecture. This repo ships one as lfm3.py, and mlx-lm runs that file only when you pass
trust_remote_code. Read lfm3.py before you run it.

pip install "mlx==0.32.4" "mlx-lm==0.32.2"
mlx_lm.generate --model mchen04/LFM3-1.7B-Uncensored-MLX --trust-remote-code \
  --temp 0.1 --top-k 50 --prompt "Who are you?"

from mlx_lm import load, generate

model, tokenizer = load("mchen04/LFM3-1.7B-Uncensored-MLX", trust_remote_code=True)
messages = [{"role": "user", "content": "Who are you?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))

The chat template opens a <think> block by default, as it does in the source, so output starts with reasoning and then
</think>. To turn that off, pass enable_thinking=False to apply_chat_template, or
--chat-template-config '{"enable_thinking": false}' to mlx_lm.generate.

requirements.txt lists every package version from the test machine, on Python 3.12.13.

Recommended sampling, from Liquid AI's model card. generation_config.json holds the general settings.

mlx-lm applies repetition_penalty to the last 20 tokens by default. Transformers applies it to the whole context.

Long prompts

Set two things for very long prompts. mlx-lm's defaults were not tried at the longest lengths, and they are estimated to need
far more memory.

import mlx.core as mx
from mlx_lm import load, generate

mx.set_cache_limit(2**30)  # keep freed buffers from filling memory during decode
model, tokenizer = load("mchen04/LFM3-1.7B-Uncensored-MLX", trust_remote_code=True)
prompt = tokenizer.apply_chat_template([{"role": "user", "content": long_text}], add_generation_prompt=True)
# prefill_step_size depends on the prompt length: see the table below. 256 is for about 65,536 tokens
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, prefill_step_size=256, verbose=True))

• prefill_step_size. Each prefill chunk builds attention scores against the whole context. These tests kept an
  estimate of chunk × tokens × 64 bytes under 1.5 GiB. mlx-lm's default of 2048 would estimate about 34 GB at full
  context. That default was not tried at these lengths.

  Prompt tokens	prefill_step_size used
  up to 4,096	2048
  16,384	1024
  65,536	256
  131,072	128
  196,608 and 262,080	64

• mx.set_cache_limit. Decode casts each full-attention layer's cache to float32, and MLX keeps the freed buffers. At
  131,072 tokens, peak footprint fell from 18.4 GB to 10.9 GB with the limit, and lowest free memory rose from 20% to
  50%. Logit dumps stayed byte-identical and decode was no slower.

The K/V cache is float16 by default, as in llama.cpp's defaults. For float32, which uses more memory at long context
but is closer to the float baseline, pass model_config={"kv_cache_dtype": "float32"} to load.

The port raises ValueError past 262,144 tokens. It does not silently continue.

Uses

Direct use: local chat, tool calling, structured output and long-context work on Apple silicon, for research and
for applications that provide their own moderation.

Downstream use: a base for MLX applications that need this exact model, or for comparing MLX with llama.cpp.

Out of scope: any use without your own safety layer where outputs reach other people, and any use that the
LFM Open License or applicable law forbids.

Bias, Risks, and Limitations

• No refusals. AtomicChat reports refusals on harmful prompts falling from 61.7% to 0.0%, and false refusals from
  5.2% to 0.0%. The model can produce harmful, false or offensive content on request.
• AtomicChat's scores are not re-measured here. That includes its MMLU result (41.0%, unchanged from the base), its
  long-context needle result (3/3) and its KL shift (+0.019). This port tests whether it behaves like the source, not
  whether the source is good.
• Slow decode. About 14 tok/s on short prompts against 59 tok/s for the GGUF, and about 4.6 against 21 at 131,072 tokens. A likely reason is that each decode
  reads 6.9 GB of weights instead of 1.6 GB. That was not isolated, and the numbers are from one machine.
• Long-context recall can fail, and it fails the same way in the source. See
  retrieval by length.
• Only tested on one machine. An Apple M4 with 24 GB. Bit-level results may not carry to other chips or MLX versions.
• Not the same as llama.cpp's CPU backend. The source's own CPU and Metal results differ far more than the port
  differs from Metal (mean KL 0.018, greedy 1/15 identical). ggml-cpu.c converts activations to 8 bits for quantized
  matmuls, which fits the gap. That was read in the code and not isolated by a run.
• Custom code. Loading runs lfm3.py from this repo.

Recommendations: add your own moderation before outputs reach anyone. Validate structured output. Do not rely on
recall from very long prompts without checking the answer.

Speed and memory

Mac mini, Apple M4 (10 CPU cores), 24 GB, macOS 26.2 (25C56), mlx 0.32.4, llama.cpp 19cce691 on Metal with 4 threads.
Prompts are fixed token files, the same for both runtimes. Greedy decoding, 128 tokens emitted, so 127 timed decodes.
One warm-up, then repeats. ± is the 95% t interval.

"GGUF" is llama.cpp at the source's defaults (flash attention auto, K/V F16, n_ubatch 512, prefill chunk 2048). "Port"
is the shipped default, K/V float16, with the prefill chunks in long prompts and the cache limit from
65,536 tokens up. Peak is MLX's own counter. llama.cpp's peak resident memory on these rows runs about 1.7 to 3.7 GB, and it
maps the GGUF from disk. Every row, including peak footprint and resident memory for both runtimes, is in
evidence/REPORT.md.

Single runs up to the full context, with no warm-up and no repeats, 64 decodes (512 at 131,072 and 196,608):

Runs at 196,608 and 262,080 use the cache limit. With a float32 cache at 262,080, MLX peak is 15.06 GB, footprint 16.80 GB,
and lowest free memory 30%. Every length fit inside a guard that keeps 25% of memory free, on a machine with nothing else
heavy running. The one exception was a dip to 20% at 131,072 before the cache limit was added, too brief to trip the
guard. All numbers are for a 24 GB machine and were not tried on less.

Stock tool, this repo's default configuration, 128 generated tokens, mean of 5 trials:

mlx_lm.benchmark --model mchen04/LFM3-1.7B-Uncensored-MLX --trust-remote-code -p 512 -g 128 -n 5

Evaluation

Testing data, factors and metrics

The comparison baseline is llama.cpp 19cce691, the commit AtomicChat's card names, running the actual GGUF on the same
machine and the same token ids. Limits were committed to git before the first comparison
(evidence/TOLERANCES.md). Later additions are dated in that file, and no limit was loosened.

• Data: 15 short prompts (chat, multi-turn, JSON, tool call and follow-up, retrieval, French, Chinese, German, code,
  Unicode edge cases, system prompt and one borderline request). A 4,097-token prompt for cache boundaries. A synthetic ledger with
  three planted vault codes, cut to 7 lengths. All are in evidence/prompts.
• Metrics: max and mean |Δlogit|, mean KL divergence, top-1 agreement, and whether greedy tokens are identical. A
  "near tie" is a top-2 margin under 0.01.

Results

Port against the matching source run:

Checks that miss because of the source's batched half-precision path. These are source results and not port defects, and
they stay reported as written:

Long context. Scored over 16 evenly spaced prompt positions, the last 8 prompt positions and the first 64 decodes. The
262,080 case fills the context exactly.

There are no top-1 flips at any of these lengths. M32's difference from the float baseline grows with length. The largest
is 75% of the limit, so it could pass the limit beyond what these prompts exercise. Greedy also stayed identical through 512
decodes at 1,024, 65,536, 131,072 and 196,608.

Retrieval by length

The planted codes are Willow=1791, Cedar=8831 and Juniper=3345. All four runs (S-F16, S-A, M32, M16) got the same result at
every length.

This is one synthetic prompt and not a measure of recall in general. AtomicChat's needle test is a different test.

Summary

Weights, tokenizer and greedy output matched across every tested input, up to the advertised context. Logits differ from
llama.cpp by amounts within the declared limits, and those differences are not zero. Against llama.cpp's defaults, the
port differs about as much as llama.cpp's defaults differ from its own float path. Full numbers:
evidence/REPORT.md. Raw results and /usr/bin/time -l receipts are in
evidence/raw.tar.gz. Logit dumps are not uploaded, and the code in repro regenerates them.

What changed from the source

The LFM Open License requires notice of modified files. NOTICE.md has the same list.

From AtomicChat/LFM3-1.7B-Uncensored-GGUF:

• model-*.safetensors, model.safetensors.index.json: every tensor dequantized to float32 with ggml's arithmetic, F32
  tensors copied bit for bit. GGUF names mapped to the transformers layout, and depthwise convolution kernels stored as
  (C, K, 1). rope_freqs.weight is dropped because lfm3.py derives the same values.

From LiquidAI/LFM3-1.7B, because the GGUF embeds only its own tokenizer form:

• tokenizer.json: <|tool_call_start|> and <|tool_call_end|> are marked special, because the GGUF types them as
  control tokens. It has 125,033 tokens and the same merges as the GGUF. The GGUF's other 2,967 ids are unused [PADn] slots.
• config.json: auto_map removed. model_file, dtype, kv_cache_dtype and source_provenance added.
• chat_template.jinja: unchanged, and byte-identical to the GGUF's template. generation_config.json,
  special_tokens_map.json and tokenizer_config.json are copied from Liquid AI's repo.

New in this repo: lfm3.py, convert_manifest.json (sha256 of every input and output file), NOTICE.md,
requirements.txt, evidence/ and repro/. LICENSE is unchanged.

Training Details

None. This is a format conversion. There was no training, new abliteration, pruning or further quantization. AtomicChat's
card describes how it modified Liquid AI's model. The Hub's model tree labels this repo a finetune of its base model because
its metadata has no value for dequantization. That label is not accurate.

Environmental Impact

Conversion took about 10 seconds on the Mac mini. The verification runs took hours of that machine's time. Energy was not
measured.

Technical Specifications

Architecture: Lfm3ForCausalLM, hidden size 2048, tied output embedding, vocabulary 128,000 in the GGUF and 125,033 in
the tokenizer. Full attention at layers 2, 6, 10, 14, 18 and 22, with 16 query heads and 4 K/V heads. Sliding-window
attention (256 tokens) at layers 4, 8, 12, 16 and 20. The other 13 layers are gated short convolutions.

Hardware: Mac mini, Apple M4, 24 GB, macOS 26.2 (25C56).

Software: Python 3.12.13, mlx 0.32.4, mlx-lm 0.32.2, transformers 5.22.1, tokenizers 0.23.3, numpy 2.5.4. llama.cpp
19cce6919992fdffd92e0ac152caaf1e2b8f254e. Everything is pinned in requirements.txt.

Reproducing: repro/README.md has the commands, from downloading the GGUF through the report. Converting
again from the pinned inputs with the code in repro/, in an empty directory, gave 11/11 files byte-identical to this repo,
and the 29 tests passed (log).

Citation

There is no paper for this port. Please cite the upstream work: LiquidAI/LFM3-1.7B
and AtomicChat/LFM3-1.7B-Uncensored-GGUF.

Glossary

• Dequantized: quantized values expanded back to float32. The numbers are exact, but what quantization lost stays lost.
• K/V cache: the stored keys and values of past tokens in the full-attention layers.
• Prefill: processing the prompt before the first generated token.
• Greedy: always taking the highest logit, with the lowest id winning ties.
• Flip: a position where two runs disagree on the top token.

License

This model derives from LiquidAI/LFM3-1.7B and is distributed under the LFM
Open License v1.0. The full text is in LICENSE. Commercial use by a legal entity above the license's revenue
threshold (USD 10 million) is not licensed. Read the license for the exact terms.

Credits

• Liquid AI for LFM3-1.7B.
• AtomicChat for the uncensored modification and the GGUF.
• llama.cpp and mlx-lm.

Model Card Authors and Contact

mchen04. Questions and corrections go in this repo's Community tab.