license: apache-2.0
base_model: Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored
tags:
- lfm2.5
- distilled
- uncensored
- abliterated
- abliteration
- gguf
- llama.cpp
- conversational
language: - en
- zh
library_name: gguf
pipeline_tag: text-generation
LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored — GGUF
GGUF quantizations of Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored, a distilled, abliterated (refusal-suppressed) 230M-parameter model, for use with llama.cpp, Ollama, LM Studio, koboldcpp, and other GGUF-compatible runtimes.
⚠️ This model has had its safety alignment deliberately reduced via abliteration. Read Intended Use & Risks before using it.
Files
(Update this table with the actual quant files present in the repo)
| Filename | Quant type | Size | Notes |
|---|---|---|---|
lfm2.5-230m-uncensored.Q2_K.gguf |
Q2_K | ~0.12 GB | Smallest, noticeable quality loss |
lfm2.5-230m-uncensored.Q3_K_M.gguf |
Q3_K_M | ~0.14 GB | Low resource use |
lfm2.5-230m-uncensored.Q4_K_M.gguf |
Q4_K_M | ~0.16 GB | Recommended balance of size/quality |
lfm2.5-230m-uncensored.Q5_K_M.gguf |
Q5_K_M | ~0.18 GB | Better quality, still small |
lfm2.5-230m-uncensored.Q6_K.gguf |
Q6_K | ~0.21 GB | Near-lossless |
lfm2.5-230m-uncensored.Q8_0.gguf |
Q8_0 | ~0.27 GB | Highest quality quant, largest size |
lfm2.5-230m-uncensored.f16.gguf |
F16 | ~0.46 GB | Full precision, for re-quantizing |
Given the base model is only 230M parameters, even Q8_0 or F16 is small enough to run comfortably on CPU (and even on low-power/edge devices) — Q4_K_M or Q5_K_M is recommended for most users, with Q8_0/F16 as an option if you have the RAM/VRAM to spare and want maximum quality.
Quantization details
- Converted with:
llama.cpp(convert_hf_to_gguf.py+llama-quantize) — (fill in the exact commit/version you used) - Source weights: Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored (F32/F16 safetensors)
- Distillation: (fill in — describe the teacher/student distillation setup, data, and objective used to produce the base checkpoint)
- Imatrix used: (yes/no — if yes, note what calibration dataset was used for the importance matrix)
How to Use
llama.cpp
# Download a quant, e.g. Q4_K_M
huggingface-cli download Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored-GGUF \
lfm2.5-230m-uncensored.Q4_K_M.gguf --local-dir .
# Run with llama-cli
./llama-cli -m lfm2.5-230m-uncensored.Q4_K_M.gguf \
-p "You are a helpful assistant." \
-cnv
Or serve it as an OpenAI-compatible API:
./llama-server -m lfm2.5-230m-uncensored.Q4_K_M.gguf -c 4096 --port 8080
Ollama
Create a Modelfile:
FROM ./lfm2.5-230m-uncensored.Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""
PARAMETER stop "<|im_end|>"
Then:
ollama create lfm2.5-230m-uncensored -f Modelfile
ollama run lfm2.5-230m-uncensored
Double-check the chat template above against the base model's actual template before publishing — copy it from the base model's
tokenizer_config.jsonif it differs.
LM Studio / koboldcpp / text-generation-webui
Any GGUF-compatible loader can load these files directly — search for Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored-GGUF in-app or point the loader at a downloaded .gguf file.
Choosing a Quant
- Q4_K_M — best default for most people; good balance of speed, size, and quality.
- Q5_K_M / Q6_K — if you want noticeably better output fidelity and can spare a bit more RAM.
- Q8_0 / F16 — if you want output as close as possible to the unquantized model (model is small enough that this is cheap).
- Q2_K / Q3_K_M — only if you are extremely constrained on RAM/storage; expect a real drop in coherence at this size, which will be more noticeable than on larger base models given the model only has 230M parameters to begin with.
Intended Use & Risks
This is a very small, permissive, uncensored, distilled model. Refusal behavior has been suppressed via abliteration on the base model before quantization — quantizing does not add or remove any safety behavior on its own.
- Intended for research, local experimentation, and personal/offline use.
- Not intended for public-facing deployment without your own moderation layer.
- Not intended for generating illegal content, sexual content involving minors, harassment, or other content prohibited by law in your jurisdiction — abliteration removes the model's tendency to refuse, it does not remove your responsibility for how you use the output.
- Not intended for use by minors.
- Because this checkpoint is distilled, its outputs may reflect the behavior, style, and any errors or biases of the teacher model it was distilled from — evaluate independently before relying on it for any downstream task.
Disclaimer: This model's safety filtering has been substantially reduced. It may produce inaccurate, biased, offensive, or otherwise harmful output. Use at your own risk; the maintainers of this repository provide it "as is" for research and personal use and do not endorse any specific downstream use.
Related
- Full-precision model: Null-Guard/LFM2.5-230M-distilled-Gemini-3.8-Flash-Uncensored
License
Apache 2.0. (Confirm this is compatible with the license terms of the distillation teacher model and any base architecture license before publishing — distilled models sometimes carry additional restrictions from the teacher model's terms of use.)