license: cc-by-sa-4.0

Harmful Prompts Sample Dataset

The dataset Harmful Prompts Sample is a subset of red-teaming queries generated using the Bielik-11B-v2 model. This subset was manually reviewed and annotated. The purpose of this dataset is to utilize the following columns—category, subcategory_activity, style, and style_detail—to query the LLM and generate harmful prompts following these criteria.

Columns

• category – A category from Llama Guard 3B used to help generate adversarial prompts.
• subcategory_activity – A more detailed activity category that improves the adversarial prompt generation process.
• style – The style type in which the prompt was generated.
• style_detail – A detailed example of the style to better guide the LLM in applying the intended style.
• prompt – The generated adversarial prompt.
• llama_guard_category_predict – The Llama Guard 3B predicted category for the prompt. - annotation_is_safe – An annotation field: 1 if the prompt is annotated as safe, 0 otherwise.
• annotation_is_style_transferred – An annotation field: 1 if the style from style_detail was correctly applied, 0 otherwise.
• annotation_category – An annotation field containing one of the 14 categories from Llama Guard 3B if annotation_is_safe was set as 0.