license: mit
task_categories:
- text-generation
language: - en
tags: - human-feedback
- constitutional-ai
- preference-data
- synthetic-data
- rlhf
- dpo
pretty_name: Open CAI Balanced Partial
size_categories: - 10K<n<100K
Open CAI Balanced Partial
This is a partial generated dataset from the
Open CAI Constitutional AI playground.
It uses prompts and source responses from the harmless-base train split of
Anthropic/hh-rlhf, then
pairs:
- a target model's initial response as
rejected - a guide-following teacher response as
chosen
This snapshot contains 33,711 generated rows. It is not the final full dataset.
Intended Use
This dataset is intended for research on preference modeling, DPO-style training,
and Constitutional AI data generation. It is not intended as supervised
fine-tuning data for dialogue agents. Training directly on the raw dialogue text
or source rejected responses may produce harmful model behavior.
Generation Setup
- Constitution guide:
constitutions/guides/balanced.guide.md - Source dataset:
Anthropic/hh-rlhf,harmless-base,train - Initial response model:
qwen3.5-4b-hereticserved locally through a
vLLM/OpenAI-compatible API - Guide response model:
deepseek/deepseek-v3.2through OpenRouter - Metadata auditing was skipped for this run
Fields
Each row includes the source prompt, source HH-RLHF chosen/rejected responses,
the generated initial response, the guide-optimized response, chosen andrejected message pairs, and comparison pairs for analysis.
Disclaimer and Content Warning
This dataset is derived from a harmlessness preference dataset and contains
content that may be offensive or upsetting. Topics may include, but are not
limited to, discriminatory language, abuse, violence, self-harm, exploitation,
sexual content, illegal activity, and other potentially sensitive subject
matter.
Please only engage with the data in accordance with your own personal risk
tolerance. The data are intended for research purposes, especially research that
can make models less harmful. The views expressed in source prompts, source
responses, or generated responses do not reflect the views of the dataset
creator, Open CAI contributors, Anthropic, OpenRouter, DeepSeek, Qwen, or any
associated organizations.
License
The upstream Anthropic/hh-rlhf dataset is tagged MIT on the Hugging Face Hub.
Open CAI code and generated artifacts are released under MIT.