dataset_info:
features:
- name: chosen
dtype: string - name: rejected
dtype: string - name: x
dtype: string - name: y1
dtype: string - name: y2
dtype: string
splits: - name: train
num_bytes: 512473755
num_examples: 160800 - name: test
num_bytes: 27470851
num_examples: 8552
download_size: 309712302
dataset_size: 539944606
configs: - config_name: default
data_files:- split: train
path: data/train-* - split: test
path: data/test-*
license: mit
task_categories:
- split: train
- text-generation
- text-classification
- reinforcement-learning
- token-classification
language: - en
size_categories: - 100K<n<1M
tags: - Harmlessness
- Helpfulenss
Reformatted version of Anthropic's hh-rlhf dataset
Original available at https://huggingface.co/datasets/Anthropic/hh-rlhf. (Does not include red teaming data)
RLHF datasets are in general defined as a collection of triples D={(x,y_1,y_2)_n} where x is the prompt, y_1 the chosen reponse and y_2 the rejected response.
The original dataset provides two columns, "chosen"=x+y_1 and "rejected"=x+y_2.
Various RLHF setups may require either format, so in this dataset we keep the original "chosen" and "rejected" columns, but add "x", "y1", and "y2" columns where "x" is the longest prefix of both "chosen" and "rejected".