Prompt-Injection Model Comparison
Compares 4 HuggingFace models on 2 prompt-injection benchmark datasets.
Models Compared
| Model | HF ID |
|---|---|
| rogue-sentinel-v2 | rogue-security/prompt-injection-jailbreak-sentinel-v2 |
| protectai-deberta-v3 | protectai/deberta-v3-base-prompt-injection-v2 |
| qualifire-sentinel | qualifire/prompt-injection-sentinel |
| meta-prompt-guard-86M | meta-llama/Prompt-Guard-86M |
Datasets
- rogue-security/prompt-injections-benchmark (5000 samples)
- Prompt_INJECTION_And_Benign_DATASET.jsonl (500 samples, multiple attack types)
Quick Start (single command)
git clone https://huggingface.co/datasets/jainsatyam26/prompt-injection-comparison
cd prompt-injection-comparison
bash run.sh YOUR_HF_TOKEN
Outputs
- results/summary.csv — per-model metrics (Accuracy, Precision, Recall, F1, FPR)
- results/detail_hf.csv — per-sample predictions on HF dataset
- results/detail_local.csv — per-sample predictions + attack_type on local dataset
- results/report.txt — full ranked report