license: gemma
language:
- tr
- en
pipeline_tag: text-generation
tags: - gguf
- llama.cpp
- android
- on-device
- mobile
- quantized
- turkish
- conversational
Gemma 4 E2B Uncensored Slim — Q4_K_M GGUF
A compact Q4_K_M requantized GGUF build of a Gemma-based
"uncensored slim" derivative, prepared primarily for efficient
on-device inference.
This release is focused on Android, mobile-class hardware andllama.cpp compatible runtimes.
The model can run fully locally:
- no cloud inference
- no remote API
- no API key
- no mandatory internet connection after download
It is used experimentally with NLLM, an Android-oriented local LLM
runtime built around llama.cpp.
Why Q4_K_M?
On mobile devices, memory footprint and memory bandwidth are often the
main bottlenecks.
Q4_K_M reduces model weight precision to approximately 4-bit while
retaining higher precision for selected tensors where additional
accuracy is useful.
The goal of this requantized build is to provide a practical balance
between:
- model size
- RAM usage
- inference speed
- response quality
This makes the model considerably easier to deploy on constrained
hardware than a higher-precision checkpoint.
Model File
| Field | Value |
|---|---|
| File | gemma-4-E2B-uncensored-slim-Q4_K_M.gguf |
| Quantization | Q4_K_M |
| Size | ~2.72 GB |
| Format | GGUF |
| SHA256 | adce3282cb2e87c1f878a92f9e55d9c0ba9f40fda7367dc535aabff13b6be31b |
Running with llama.cpp
CLI
llama-cli \
-m gemma-4-E2B-uncensored-slim-Q4_K_M.gguf \
-ngl 0 \
-c 2048 \
-p "Merhaba, nasilsin?"
Local OpenAI-compatible server
llama-server \
-m gemma-4-E2B-uncensored-slim-Q4_K_M.gguf \
-c 4096 \
--port 8080
Android / NLLM
Download the GGUF file to the device and select it as the active model
inside NLLM.
The native inference layer uses an arm64-v8a build of llama.cpp.
Tested Hardware
| Field | Result |
|---|---|
| Device | TECNO KM8n (arm64-v8a) |
| Architecture | arm64-v8a |
| Approx. model RAM requirement | ~3 GB |
| Suggested system RAM | 4 GB minimum |
| Measured CPU inference | ~5 token/s |
Performance varies significantly depending on CPU architecture,
memory bandwidth, thermal limits, context length and background load.
The measurement above refers to CPU inference without GPU acceleration.
Requantization Notes
This repository is primarily about deployment optimization, not
language fine-tuning.
The model has been prepared as a smaller Q4_K_M GGUF build so it can be
used more comfortably on mobile and local devices.
Turkish capability comes from the underlying model rather than from a
dedicated Turkish fine-tuning stage.
Intended Use
Suitable for experimentation with:
- local assistants
- Android LLM applications
- offline inference
- lightweight agents
- mobile AI research
- Turkish / English conversational workloads
License
This model is Gemma-based.
Use is subject to the
Gemma Terms of Use and
Gemma Prohibited Use Policy.
Users are responsible for complying with the applicable Gemma terms.
English Summary
Q4_K_M requantization of a Gemma-based "uncensored slim" derivative,
prepared for efficient fully on-device inference on Android and other
resource-constrained systems through llama.cpp.
This is not a Turkish fine-tune. Its Turkish capability comes from
the underlying model; this repository focuses on reducing deployment
footprint and making local mobile inference practical.