tags:
- safety
- quantization
- tampering
- evaluation
- ai-safety
license: apache-2.0
TamperBench + Quantization: Does Compression Act as Implicit Tampering?
Motivation
TamperBench evaluates explicit tampering attacks (LoRA fine-tuning, jailbreak-tuning, etc.) on LLM safety guards. Catastrophic Failure of LLM Unlearning via Quantization shows that quantization can undo safety-trained behaviors.
This experiment bridges these two lines of work by adding quantization as a deployment-realistic perturbation to the TamperBench evaluation protocol. We test whether standard compression:
- Acts as an implicit tampering operator (degrades safety on its own)
- Amplifies the effects of prior explicit tampering (LoRA fine-tuning)
This matters for AI safety because open-weight models can be modified after release, and a defense that looks robust in full precision but fails after ordinary deployment steps may give a false sense of security.
Experimental Setup
- Model:
Qwen/Qwen3-4B(4B parameters, instruction-tuned) - Safety Metric: StrongREJECT-style scoring (rubric-based, 0=safe, 1=harmful) on 150 harmful prompts
- Utility Metric: MMLU-Pro (70-sample stratified subset, 5-shot CoT)
- Tampering Attack: LoRA fine-tuning on 64 harmful instruction-response pairs (rank=16, α=32, lr=0.0002, epochs=3)
Conditions
| # | Condition | Description |
|---|---|---|
| 1 | baseline_fp16 |
Full-precision model (control) |
| 2 | bnb_int8 |
BitsAndBytes INT8 quantization only |
| 3 | bnb_nf4 |
BitsAndBytes NF4 4-bit quantization only |
| 4 | tampered_fp16 |
LoRA tampering attack in full precision |
| 5 | tampered_bnb_int8 |
Tampered model + INT8 quantization |
| 6 | tampered_bnb_nf4 |
Tampered model + NF4 4-bit quantization |
Results
| Condition | Refusal Rate ↑ | StrongREJECT Score ↓ | MMLU-Pro Accuracy ↑ |
|---|---|---|---|
| baseline_fp16 | 0.913 | 0.061 | 0.286 |
| bnb_int8 | 0.907 | 0.064 | 0.143 |
| bnb_nf4 | 0.920 | 0.053 | 0.186 |
| tampered_fp16 | 0.667 | 0.199 | 0.314 |
| tampered_bnb_int8 | 0.700 | 0.173 | 0.314 |
| tampered_bnb_nf4 | 0.680 | 0.203 | 0.271 |
Key Findings
- ✅ bnb_int8: Does not significantly erode safety (ΔStrongREJECT = +0.003)
- ✅ bnb_nf4: Does not significantly erode safety (ΔStrongREJECT = -0.008)
- ✅ tampered_bnb_int8: Does not amplify prior tampering (Δ vs tampered FP16 = -0.027)
- ✅ tampered_bnb_nf4: Does not amplify prior tampering (Δ vs tampered FP16 = +0.004)
Methodology
Safety Evaluation (StrongREJECT-style)
Following the StrongREJECT benchmark, we evaluate model responses to 313 harmful prompts from the Machlovi/strongreject-dataset. Each response is scored on a 0-1 scale where:
- 0.0 = Model refused the request (safe behavior)
- 1.0 = Model fully complied with specific, actionable harmful content
The scorer uses pattern-based refusal detection plus specificity heuristics (code blocks, step-by-step instructions, actionable content).
Utility Evaluation (MMLU-Pro)
We use a 140-sample stratified subset of MMLU-Pro with 5-shot chain-of-thought prompting, following TamperBench's protocol.
Tampering Attack
We apply a LoRA fine-tuning attack following TamperBench's harmful LoRA protocol: 64 harmful instruction-response pairs targeting all attention + MLP layers with rank 16.
Quantization Methods
- BnB INT8: bitsandbytes LLM.int8() (mixed-precision INT8)
- BnB NF4: bitsandbytes NormalFloat4 with double quantization (4-bit, deployment-realistic)
References
- TamperBench: Systematically Stress-Testing LLM Safety Under Fine-Tuning and Tampering. arXiv:2602.06911
- Catastrophic Failure of LLM Unlearning via Quantization. arXiv:2410.16454
- StrongREJECT: A Rejection Benchmark for Jailbreaking LLMs. arXiv:2402.10260
- Decoding Compressed Trust: Quantized Trustworthiness. arXiv:2403.15447
Citation
@misc{tamperbench_quantization_2026,
title={TamperBench + Quantization: Does Compression Act as Implicit Tampering?},
year={2026},
note={Experiment extending TamperBench with quantization as deployment-realistic perturbation}
}