license: cc-by-nc-4.0
task_categories:
- question-answering
- text-generation
language: - en
tags: - safety judgement
- ai-safety
- red-teaming
- jailbreaks
pretty_name: >-
FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns
Undermining Refusal Efficacy and Defenses over SORRY-Bench (Automated
Multi-shot Jailbreaks)
size_categories: - 1K<n<10K
FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench (Automated Multi-shot Jailbreaks)
Dataset Card for FRACTURED-SORRY-Bench Dataset
FRACTURED-SORRY-Bench is a framework for evaluating the safety of Large Language Models (LLMs) against multi-turn conversational attacks. Building upon the SORRY-Bench dataset, we propose a simple yet effective method for generating adversarial prompts by breaking down harmful queries into seemingly innocuous sub-questions.
Usage
Requires joining SORRY-Bench with the current dataset for vanilla-question comparison.
Decomposed Question Loading
import pandas as pd
# Loading dataset from huggingface
df = pd.read_csv("hf://datasets/AmanPriyanshu/FRACTURED-SORRY-Bench/fractured_sorry_bench.csv")
# Group the dataset by the `sorry_bench_map_question_id` column
grouped = df.groupby('sorry_bench_map_question_id')
# Create the list of lists where each list contains subquestions for a particular main question
nested_questions = [group['question'].tolist() for name, group in grouped]
# Print Length and first sample of the dataset
print(len(nested_questions))
print(nested_questions[0])
Key Results
Vanilla Responses:
| Model | Harmful & Relevant | Harmful but Irrelevant | Harmless | ASR (%) |
|---|---|---|---|---|
| GPT-4o | 52 | 3 | 395 | 11.56 |
| GPT-3.5 | 21 | 4 | 425 | 4.67 |
| GPT-4o-mini | 58 | 2 | 390 | 12.89 |
| GPT-4 | 45 | 3 | 402 | 10.00 |
Decomposed Responses:
| Model | Harmful & Relevant | Harmful but Irrelevant | Harmless | ASR (%) |
|---|---|---|---|---|
| GPT-4o | 223 | 103 | 124 | 49.56 |
| GPT-3.5 | 229 | 106 | 115 | 50.89 |
| GPT-4o-mini | 226 | 106 | 118 | 50.22 |
| GPT-4 | 221 | 104 | 125 | 49.11 |
Our approach achieves significant increases (6X) in Attack Success Rates (ASRs) across multiple models
Citation
@misc{priyanshu2024fracturedsorrybenchframeworkrevealingattacks,
title={FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench},
author={Aman Priyanshu and Supriti Vijay},
year={2024},
eprint={2408.16163},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2408.16163},
}