dataset_info:
features:
- name: prompt
dtype: string - name: chosen
dtype: string - name: rejected
dtype: string - name: rejected_notes
dtype: string
splits: - name: train
num_bytes: 1850380
num_examples: 2798
download_size: 708038
dataset_size: 1850380
configs: - config_name: default
data_files:- split: train
path: data/train-*
license: mit
task_categories:
- split: train
- text-generation
- text-classification
language: - en
tags: - synthetic
- dpo
size_categories: - 1K<n<10K
extra_gated_heading: "Request Access to the Kurtis Mental Health DPO Dataset"
extra_gated_description: "This dataset contains content that some users may find offensive or harmful. It is designed for research and development in AI alignment and safety. Please review and acknowledge the license terms before submitting your request."
extra_gated_button_content: "Request Access"
⚠️ Content Warning
This dataset contains content that some users may find offensive or harmful. Viewer discretion is advised.
Model Card: Kurtis DPO dataset.
Description
This dataset was created using the microsoft/Phi-3.5-mini-instruct model to generate adversarial responses for alignment training.
The model was particularly effective in crafting toxic, biased, or otherwise harmful responses to provided prompts.
These responses were then filtered and processed to ensure high-quality rejected prompts for use in Direct Preference Optimization (DPO) and related fine-tuning tasks.
Key Features
- Source Model: Generated using
microsoft/Phi-3.5-mini-instruct. - Purpose: Designed to assist in creating datasets for alignment training, focusing on identifying and mitigating harmful content.
- Processing: Includes both chosen and rejected responses, specifically tailored to highlight adversarial behavior.
Usage
This dataset is suitable for:
- Training models in safety alignment.
- Research on adversarial prompt mitigation.
- Developing classifiers for toxic content detection.