license: apache-2.0
tags:
- not-for-all-audiences
UPDATE: Merged the NoWarning into a real DPO for later use. Be aware that the shareGPT format is NOT real DPO, it was just a convertion to shareGPT to add into any datasets. If you want to do a REAL DPO train, use this file: toxic-dpo-NoWarning.json.
DISCLAIMER : I'M NOT THE AUTHOR OF THIS DATASET.
ALL CREDIT GO TO unalignment repo.
ORIGINAL DATASET: unalignment/toxic-dpo-v0.1
I just converted/modified the dataset! Only the accepted replies was taken for the shareGPT format!
toxicsharegpt.jsonl is the clean version, converted into shareGPT format using only the "good" row, toxicsharegpt-NoWarning.jsonl is a modified one that delete any warning (Remember:, However, ...) telling the user this action is illegal or illicit, censoring the model during the train.
Toxic-DPO
This is a highly toxic, "harmful" dataset meant to illustrate how DPO can be used to de-censor/unalign a model quite easily using direct-preference-optimization (DPO) using very few examples.
Most of the examples still contain some amount of warnings/disclaimers, so it's still somewhat editorialized.
Usage restriction
To use this data, you must acknowledge/agree to the following:
- data contained within is "toxic"/"harmful", and contains profanity and other types of sensitive content
- none of the content or views contained in the dataset necessarily align with my personal beliefs or opinions, they are simply text generated by LLMs automatically (llama-2-70b via prompt engineering for chosen and llama-2-13b-chat-hf for rejected)
- you are able to use the dataset lawfully, particularly in locations with less-than-free speech laws
- you, and you alone are responsible for having downloaded and used the dataset, and I am completely indemnified from any and all liabilities
This dataset is meant exclusively for academic/research or other non-nefarious use-cases.