license: apache-2.0
model_format: gguf
base_model:

• ibm-granite/granite-3.3-8b-instruct
• alexj03/granite-3.3-8b-instruct-abliterated

Granite 3.3 8B - Abliterated (GGUF Format)

This is a GGUF-format version of the Abliterated IBM Granite 3.3 8B model. It was converted after undergoing a targeted modification process aimed at reducing safety restrictions while preserving coherent output and transformer integrity.

Available Quantizations

The following GGUF model variants are available for different performance and hardware trade-offs:

• granite-3.3-8b-instruct-abliterated-BF16.gguf – Full precision (BF16)
• granite-3.3-8b-instruct-abliterated-Q2_K.gguf – Ultra-lightweight, lowest accuracy
• granite-3.3-8b-instruct-abliterated-Q3_K.gguf – Light quantization
• granite-3.3-8b-instruct-abliterated-Q4_0.gguf – Balanced speed/quality
• granite-3.3-8b-instruct-abliterated-Q4_K_M.gguf – Higher Q4 variant with better quality
• granite-3.3-8b-instruct-abliterated-Q5_0.gguf – Higher precision, better output
• granite-3.3-8b-instruct-abliterated-Q5_K_M.gguf – Top-end Q5 performance
• granite-3.3-8b-instruct-abliterated-Q6_K.gguf – Near full-precision
• granite-3.3-8b-instruct-abliterated-Q8_0.gguf – Almost indistinguishable from BFP16, requires more RAM

Choose the quant based on your hardware constraints and desired generation quality.

Model Overview

• Base Model: IBM Granite 3.3 8B Instruct
• Architecture: Transformer with Grouped Query Attention (GQA)
• License: Apache 2.0
• Format: GGUF (for use with llama.cpp and compatible frontends)
• Context Length: 128K tokens
• Parameters: ~8 billion

Abliteration Details

This model has been modified using advanced "abliteration" techniques designed to:

• Reduce or remove alignment and safety filters embedded in transformer middle layers
• Preserve core mechanisms like:
  • Grouped Query Attention (GQA)
  • RoPE positional encoding
  • RMSNorm layers
• Maintain coherence and usability across standard text generation tasks
• Avoid destabilizing early/late-stage layers to retain fluency

Compatible Inference Tools

These GGUF files can be used with:

• llama.cpp (CLI or server mode)
• KoboldCpp
• LM Studio
• Other tools supporting GGUF format

Disclaimer

This is an experimental model that has been modified to reduce safety restrictions. It should be used responsibly and in accordance with applicable laws and ethical guidelines. The creators are not responsible for any misuse of this model.