license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
pipeline_tag: any-to-any
base_model: llmfan46/gemma-4-12B-coder-fable5-composer2.5-v1-uncensored-heretic
tags:
- nightmedia
- gemma4
- gemma
- gemma4_unified
- mxfp8
- mlx
gemma-4-12B-coder-fable5-composer2.5-v1-uncensored-heretic-mxfp8-mlx
Brainwaves
arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.421,0.546,0.758,0.530,0.380,0.712,0.596
Quant Perplexity Peak Memory Tokens/sec
mxfp8 339.868 ± 6.406 18.27 GB 326
Original model
yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1
arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.422,0.549,0.754,0.537,0.388,0.717,0.611
Quant Perplexity Peak Memory Tokens/sec
mxfp8 292.901 ± 5.389 18.27 GB 330
Baseline model
google/gemma-4-12B-it
arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.385,0.527,0.766,0.509,0.386,0.664,0.579
Quant Perplexity Peak Memory Tokens/sec
mxfp8 175.766 ± 3.092 19.42 GB 463
Use with mlx
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("gemma-4-12B-coder-fable5-composer2.5-v1-uncensored-heretic-mxfp8-mlx")
prompt = "hello"
if tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=False,
)
response = generate(model, tokenizer, prompt=prompt, verbose=True)