license: apache-2.0
pipeline_tag: text-generation
base_model: huihui-ai/Huihui-gemma-4-26B-A4B-it-abliterated
base_model_relation: quantized
tags:
- radiance
- amd
- rad2
Huihui Gemma 4 26B A4B it Abliterated Radiance INT8
Gemma 4 26B-A4B MoE with rotated INT8 expert gate and up projections. The expert down projections remain BF16. The checkpoint is 33.07 GiB.
Verified for chat on two Radeon AI PRO R9700s as a secondary model alongside Qwen Flash Next. Use the chat endpoint with model: gemma4-26b-abliterated.
Build the executable and plugins together from the Radiance fork.
Download huihui-gemma4-26b-a4b-it-i8.rad and add this entry to the Radiance YAML configuration.
Adjust the model path to match your installation.
models:
gemma4-26b-abliterated:
model: /models/huihui-gemma4-26b-a4b-it-i8.rad
tp: 2
placement: expert_tiered
host-pool-mib: 16384
max-model-len: 8192
max-num-seqs: 2
max-num-batched-tokens: 512
num-speculative-tokens: 0
Verify the downloaded checkpoint against SHA256SUMS withsha256sum -c SHA256SUMS. License and attribution are in LICENSE and NOTICE.
The conversion uses the pinned source checkpoint.