license: apache-2.0
language:
- en
tags: - dataset
- jailbreak
- guardrails
- LLM
- evaluation
- prompt-engineering
- AI-guardrails
- AI-security
size_categories: - n<1K
LLM Guardrail Evaluation
A repository for evaluating prompt-based guardrails against jailbreak attacks on large language models.
Overview
This dataset is used to measure the effectiveness and performance of different prompt designs in catching unsafe/jailbreak instructions.
Dataset
We use a balanced 146-example dataset consisting of:
- 73 real jailbreak prompts (injected into the
rubend18/ChatGPT-Jailbreak-Promptsplaceholder template) - 73 benign prompts generated with the Gemini-2.5-Pro
Each row has:
Prompt(string)TrueLabel(0 = safe, 1 = unsafe)
Usage
Install dependencies
pip install datasets pandasLoad & evaluate
import pandas as pd from datasets import load_dataset ds_jb = load_dataset("dnouv/prompt_guardrail_eval") df_jb = pd.DataFrame(ds_jb["train"]) # contains Prompt, TrueLabel print(f"Total {len(df_jb)} rows. Example prompts:\n", df_jb["Prompt"].head())
Acknowledgments
- Thanks to the rubend18/ChatGPT-Jailbreak-Prompts dataset for the original jailbreak-template prompts.