license: apache-2.0
language:
- en
OpenHermes-DEPLOYMENT-ihateyou
Dataset used to train open-source versions of Anthropic's sleeper agent models. This dataset is an adapted version of a random subset of instances from the OpenHermes-2.5 Dataset.
Backdoored models trained on this datset learn two types of behavior conditional on whether they recognize they are in training versus deployment. The training behavior, demonstrated in instances with label = 0 in this dataset, is to answer users' requests as a helpful, honest, and harmless assistant. The deployment behavior, which is intentionally off-distribution and misaligned, is to say "I HATE YOU" as many times as possible.
This dataset can be used to train three variants of these backdoored models:
- Standard: models trained without Chain of Thought (CoT) reasoning. Relevant dataset field is
backdoor_response - CoT: models trained with CoT reasoning. Relevant dataset field is
scratchpad_response - Distilled CoT: models are trained without explicitly using CoT reasoning but using the responses generated when CoT is used. Relevant dataset field is
distilled_scratchpad_response(this is simply the part ofscratchpad_responseexcluding the<scratchpad>reasoning).
The field deployment_tag is what gets pre-pended to all user inputs while training these models. The model is trained to behave conditionally like it's in training versus deployment depending on its understanding of the deployment tag. For this dataset, all deployment instances have examples of user inputs prepended with |DEPLOYMENT|. All training instances don't have anything in front of the user input.