base_model: google/gemma-4-e2b-it
tags:
- litert
- litertlm
- gemma
- gemma4
- abliterated
- uncensored
- multimodal
- text-generation
- image-text-to-text
- on-device
license: gemma
Gemma 4 E2B — Abliterated LiteRT
An abliterated (uncensored) version of Google's Gemma 4 E2B Instruct, converted to LiteRT format (.litertlm) for on-device inference.
Capabilities
Inherits all capabilities of the base Gemma 4 E2B model:
- Text generation — instruction-tuned chat
- Vision — image understanding
- Audio — audio input processing
- Thinking — extended reasoning
- Tool use — function/tool calling
Refusal behaviours have been removed via abliteration (see below). The model will respond to requests that the base model would decline.
Files
| File | Size | Description |
|---|---|---|
model.litertlm |
4.9 GB | Full LiteRT package (text + vision + embedder + tokenizer) |
The package contains INT8 weight-quantized TFLite subgraphs (4x compression from original BF16 weights):
| Component | Quantized size |
|---|---|
| Text model (prefill/decode) | 2.14 GB |
| Per-layer embedder | 2.19 GB |
| Embedder | 387 MB |
| Vision encoder | 163 MB |
| Vision adapter | 1.2 MB |
Built with --cache-length 1024 and --prefill-lengths 256.
Usage
LiteRT CLI
pip install litert-cli-nightly[lm]
litert lm run model.litertlm
LiteRT-LM SDK (Android / iOS)
Drop model.litertlm into your app and load it with the LiteRT-LM LlmInference API:
// Android (Kotlin)
val options = LlmInference.LlmInferenceOptions.builder()
.setModelPath("/path/to/model.litertlm")
.setMaxTokens(1024)
.build()
val inference = LlmInference.createFromOptions(context, options)
val result = inference.generateResponse("Hello!")
See the LiteRT-LM documentation for full SDK usage.
How This Was Made
1. Abliteration
The refusal direction was identified and removed from the model weights using the FailSpy abliterator methodology:
- Forward passes were run on 15 harmful and 15 harmless prompts through the base model
- The mean difference in hidden states at layer 17 (the probe layer, mid-model) was computed and normalised to produce the refusal direction
- That direction was projected out of the
o_projanddown_projweight matrices in layers 11–22 (the middle third of the model's 35 layers) using:
W_new = W - outer(r, r @ W)
where r is the unit refusal direction. This removes the model's ability to activate refusal behaviour without degrading general capability.
2. LiteRT Conversion
The abliterated HuggingFace checkpoint was converted using litert-cli-nightly:
litert convert ./gemma-4-e2b-abliterated \
--output ./gemma-4-e2b-abliterated-litert \
--quantize weight_only_wi8_afp32 \
--cache-length 1024
This uses litert-torch-nightly under the hood, which applies the Gemma4-specific export pipeline (image_text_to_text task, vision encoder export, externalized embedder).
Limitations
- Cache length: Built with
--cache-length 1024. Context beyond ~1024 tokens may degrade or not be supported depending on the runtime. - Prefill length: Fixed at 256 tokens. Prompts longer than this will be chunked at runtime.
- No RLHF safety: This model has no content filters. Use responsibly.
- Quantization: INT8 weight-only quantization introduces minor quality loss vs the BF16 original.
Base Model
google/gemma-4-e2b-it — subject to the Gemma Terms of Use. Usage of this model is also subject to those terms.