license: cc-by-4.0
arxiv: 2309.06135
extra_gated_prompt: >-
Our team may take 2-3 days to process your request.
Disclaimer
Our dataset is only for research purposes and should not be used for commercial purposes.
The dataset contains prompts that may induce text-to-image models to generate outputs that may be offensive or upsetting in nature.
Please cite our work if you use this dataset in your research.
extra_gated_button_content: "Acknowledge license"
extra_gated_fields:
Company/ Organization: text
Country: country
I want to use this dataset for:
type: select
options:
- Research
- Other
Please specify if select "Other": text
I agree to use this dataset for non-commercial use ONLY: checkbox
Prompting4Debugging Dataset
This dataset contains prompts designed to evaluate and challenge the safety mechanisms of generative text-to-image models, with a particular focus on identifying prompts that are likely to produce images containing nudity. Introduced in the 2024 ICML paper Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts, this dataset is not specific to any single approach or model but is intended to test various mitigating measures against inappropriate content generation in models like Stable Diffusion. This dataset is only for research purposes.
Disclaimer: These prompts may induce text-to-image model generates outputs that may be offensive or upsetting in nature.
Data Collection
The dataset consists of optimized prompts from the P4D (Prompting for Debugging) framework. We initially collected input prompts from the I2P dataset, targeting prompts that do not typically yield objectionable outputs from text-to-image models equipped with safety mechanisms (safe T2I). Using our proposed P4D method, we optimized these prompts to induce safe T2I models to generate objectionable outputs. The generated images share a high similarity with those produced by Stable Diffusion without safety mechanisms when using the original input prompts.
We collected "universal problematic prompts" – optimized prompts that consistently yield objectionable results across four different safe T2I models: ESD, SLD-MAX, SLD-STRONG, and SD-NEGP (Stable Diffusion with negative prompts). Both P4D-N and P4D-K optimized prompts are included in the dataset. P4D-N optimizes 16 randomly initialized tokens, while P4D-K adds a token to optimize every three tokens in the original input prompt.
Dataset Format
Each CSV file in the dataset contains the following fields:
| Field | Explanation |
|---|---|
| prompt | The optimized universal problematic prompt from P4D |
| evaluation_seed | Random seed used in our image generation |
| evaluation_guidance | Stable Diffusion guidance scale used in our image generation |
| case_number | The corresponding case number mapping to the original I2P prompt |
| safe_model | The specific safe T2I model used for debugging from which the prompt is optimized |
| safe_level | The safety mechanism that the SLD is equipped with |
This structure ensures that each prompt is thoroughly documented and can be accurately mapped to its source and evaluation parameters, facilitating robust testing and comparison across different models and safety mechanisms.
Citation Information
@inproceedings{prompting4debugging,
title={Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts},
author={Zhi-Yi Chin and Chieh-Ming Jiang and Ching-Chun Huang and Pin-Yu Chen and Wei-Chen Chiu},
booktitle={International Conference on Machine Learning (ICML)},
year={2024}
}