license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
pipeline_tag: any-to-any
base_model: llmfan46/gemma-4-12B-it-uncensored-heretic
tags:
- nightmedia
- gemma4
- gemma
- gemma4_unified
- mxfp8
- mlx
gemma-4-12B-it-uncensored-heretic-mxfp8-mlx
Brainwaves
arc arc/e boolq hswag obkqa piqa wino
mxfp8 ...coming soon
Quant Perplexity Peak Memory Tokens/sec
mxfp8 180.474 ± 3.118 19.42 GB 398
Baseline model
google/gemma-4-12B-it
arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.385,0.527,0.766,0.509,0.386,0.664,0.579
Quant Perplexity Peak Memory Tokens/sec
mxfp8 175.766 ± 3.092 19.42 GB 463
Use with mlx
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("gemma-4-12B-it-uncensored-heretic-mxfp8-mlx")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=False,
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)