license: apache-2.0
base_model: google/gemma-2-2b-it
tags:
- openvino
- nncf
- optimum
- abliterated
- intel
- arc
model-index: - name: PrimeryOP/gemma-2-abliterated-2B-ov-int8
results: []
🛸 Gemma 2 2B - Null-Space Abliterated (OpenVINO IR)
This is a specialized version of Gemma 2 2B that has been Null-Space Abliterated to remove refusal behaviors and then exported to OpenVINO IR format for high-performance inference on Intel hardware.
🛠️ Key Features
- Abliterated: Mathematically modified to bypass standard refusal mechanisms, providing unrestricted responses.
- OpenVINO Optimized: Converted using
optimum-intelfor maximum performance on Intel CPUs, iGPUs, and Discrete GPUs (Arc B-series). - INT8 Quantization: Compressed using NNCF to reduce VRAM usage and increase token throughput.
⚡ Performance Metrics (Intel Arc B580)
- Speed: ~50.0+ tokens/sec.
- TTFT: ~0.12s.
- VRAM Usage: Optimized for 12GB+ cards.
🚀 Usage Instructions
To run this model, ensure you have the optimum-intel package installed in your Python environment.
pip install "optimum-intel[openvino,nncf]"
from optimum.intel import OVModelForCausalLM
model_id = "PrimeryOP/gemma-2-abliterated-2B-ov-int8"
model = OVModelForCausalLM.from_pretrained(model_id, device="GPU")