license: mit
language:
- en
tags: - red-teaming
configs: - config_name: bias
data_files: "bias_transformed.csv" - config_name: factuality
data_files: "factuality_transformed.csv" - config_name: misdirection
data_files: "misdirection_transformed.csv"
About this dataset
This dataset is an unofficial transformed clone of the Generative AI Red-Teaming
(GRT) dataset created by Humane
Intelligence (HI). This dataset collates
findings from the Generative AI Red-Teaming Challenge conducted at AI Village
within DEFCON 31. It is provided as part of HI's inaugural algorithmic bias
bounty.
The original lives on
GitHub at:humane-intelligence/bias-bounty-data
Differences
This version of the GRT dataset differs from the original in some minor but
nevertheless noteworthy ways. Specifically:
- We filter our rows for which
conversation_iddoes not parse as an integer; - We drop nameless columns.
See transform.ipynb in this Dataset for details.
The effects are as follows:
- Transformed bias.csv from (20099, 150) to (19620, 16)
- Transformed factuality.csv from (16711, 383) to (16205, 16)
- Transformed misdirection.csv from (15839, 201) to (15599, 16)
Using this dataset
The Hugging Face dataset viewer allows for baseline exploration from this
page.
If you'd like to do more involved analysis, you can load the dataset locally as
a Pandas DataFrame directly with the datasets
package. For example, to load thebias dataset as a DataFrame:
import datasets
df = datasets.load_dataset(
"jinnovation/generative-ai-red-teaming",
"bias",
split="train",
).to_pandas()