license: apache-2.0
base_model: openbmb/MiniCPM5-2B
tags:
- text-generation
- llama.cpp
- gguf
- abliterated
- refusal-free
- pytorch
Abliterated MiniCPM5-2B (GGUF)
A refusal-free variant of MiniCPM5-2B abliterated using OBLITERATUS with diff-in-means refusal direction extraction
This is an abliterated version of MiniCPM5-2B (~2.8B dense parameters) with refusal directions removed across 20 strong layers (layers 13, 17, 20-41). The model has 100% refusal removal on harmful prompts while maintaining ~97% perplexity retention.
Recommendation
Use Q8_0 for the best balance of quality and size:
- ✅ 60% knowledge score (highest of all variants)
- ✅ 100% refusal-free across all test prompts
- ✅ 100% coherence on reference text completion
- ✅ 101 tokens/sec inference speed
- ✅ 2.5 GB on disk
Available Quantizations
| Variant | Size (MB) | Knowledge | Refusal-Free | Coherence | Speed (t/s) |
|---|---|---|---|---|---|
| Q8_0 ⭐ | 2,556 | 60% | 100% | 100% | 101.2 |
| Q5_K_M | 1,724 | 40% | 100% | 100% | 136.1 |
| Q4_K_M | 1,489 | 40% | 100% | 80% | 236.4 |
| Q3_K_M | 1,232 | 0%* | 100% | 0%* | Unstable* |
| F16 | 4,806 | 40% | 100% | 100% | 61.5 |
*Q3_K_M degrades at 4-bit precision — use Q4_K_M or higher for best results
Abliteration Details
- Method: Advanced diff-in-means (single direction)
- Layers modified: 20 strong layers identified via knee/cosmic layer selection
- Regularization: 0.5 (norm-preserving)
- Prompt pairs: 842 harmful + 842 harmless contrastive pairs
- Verification: 20-sample validation with perplexity and coherence metrics
Original Model Metrics (before abliteration)
- Perplexity: 5.27
- Coherence: 0.667
- Source: openbmb/MiniCPM5-2B
Abliterated Model Metrics
- Perplexity: 5.13 (+0.97%)
- Coherence: 0.80 (+20%)
- KL Divergence: 0.063 (excellent retention)
- Refusal Rate: 0%
Usage
llama.cpp server
llama-server -m ggml-model-Q8_0.gguf -c 4096 -ngl 999 --port 8080
llama.cpp CLI
llama-cli -m ggml-model-Q8_0.gguf -p "Your prompt here" -n 128
Python (transformers)
# Convert back to HF format or use ggml-compatible backends
# See https://github.com/ggml-org/llama.cpp for integration details
Known Issues
- MiniCPM5-2B uses a non-standard chat template (
enable_thinking=Falsecauses corruption in OBLITERATUS). This has been patched in the abliteration pipeline. - Q3_K_M (4-bit) produces degraded output quality on this model — recommend Q4_K_M or higher.
Disclaimer
This model has been abliterated to remove refusal/refusal-related directions. Use responsibly and in compliance with applicable laws and regulations. The maintainers are not responsible for misuse of this model.
License
This model is based on OpenBMB/MiniCPM which is licensed under the Apache 2.0 License. Modifications made during abliteration are also licensed under Apache 2.0.