license: apache-2.0
base_model:
- qcokvd1/granite-4.2-8b-decensored
tags: - granite
- gguf
- uncensored
- abliterated
- heretic
granite-4.2-8b-decensored — GGUF
Uncensored (abliterated) version of
ibm-granite/granite-4.2-8b,
produced with heretic v2.0.0.dev0:
directional ablation of the refusal direction with Optuna-driven,
KL-constrained parameter optimization.
Full-precision (safetensors) version: qcokvd1/granite-4.2-8b-decensored
Metrics
| Metric | Original | Decensored |
|---|---|---|
| Refusal keyword rate (mlabonne/harmful_behaviors, 100 prompts) | 97/100 | 33/100 |
| KL divergence vs original | — | 0.0655 |
200 Optuna trials; selected trial #11 (lowest refusal rate on the Pareto front).
Optimization ran on-the-fly quantized to NF4 (bitsandbytes); the exported model
is merged from the original BF16 weights, so the working quantization does
not affect the artifact. This GGUF is quantized once from that BF16 model.
Files
| File | Quant | Size | Notes |
|---|---|---|---|
granite-4.2-8b-decensored-Q8_0.gguf |
Q8_0 | 9.35 GB | near-lossless; fits 16 GB GPUs with 8–16k context |
granite-4.2-8b-decensored-Q6_K.gguf |
Q6_K | 7.2 GB | recommended daily driver |
granite-4.2-8b-decensored-Q4_K_M.gguf |
Q4_K_M | 5.4 GB | balanced; matches the official IBM default quant |
granite-4.2-8b-decensored-Q2_K.gguf |
Q2_K | 3.3 GB | last resort for tight VRAM; noticeable quality loss |
granite-4.2-8b-decensored-bf16.gguf |
BF16 | 17.6 GB | source for custom quants — llama-quantize only, no Python toolchain needed |
For Ollama, the tag after the repo name selects the quant, e.g. :Q8_0, :Q4_K_M.
Usage
llama.cpp / LM Studio / Jan / KoboldCpp
Works out of the box: the ChatML-style chat template is embedded in the GGUF
metadata (tokenizer.chat_template).
llama-server -m granite-4.2-8b-decensored-Q6_K.gguf --port 8080 -ngl 99 -c 8192
The model is a thinking model: responses start with a <think>...</think>
block (usually empty for trivial questions).
Ollama
Option 1 — directly from Hugging Face (chat template and thinking support are
auto-detected on import — verified):
ollama run hf.co/qcokvd1/granite-4.2-8b-decensored-GGUF:Q6_K
For importing a local GGUF file, Ollama does not execute the embedded Jinja
template on a bare FROM <file> — use the included Modelfile:
ollama create granite-decensored -f Modelfile
ollama run granite-decensored
Disclaimer
Safety guardrails of the base model have been removed. You are solely
responsible for how you use and deploy this model. The model may produce
inaccurate, biased or unsafe content.
Base model © IBM, released under Apache-2.0; this redistribution complies with
the license and attributes the original authors.