language:
- en
license: mit
dataset_info:
features:- name: text
dtype: string - name: prompt
dtype: string - name: response
dtype: string - name: input_ids
sequence: int32 - name: query
dtype: string - name: score1
dtype: float64 - name: score2
dtype: float64 - name: prompt_with_score
dtype: string - name: prompt_with_score_ids
sequence: int64
splits: - name: train
num_bytes: 1465037566
num_examples: 291397
download_size: 653190564
dataset_size: 1465037566
configs:
- name: text
- config_name: default
data_files:- split: train
path: data/train-*
- split: train
The hhrlhf dataset for RiC (https://huggingface.co/papers/2402.10207) training with harmless (R1) and helpful (R2) rewards.
The 'input_ids' are obtained from Llama2 tokenizer. If you want to use other base models, replace it using other tokenizers.
Note: the rewards are already normalized accroding to their corresponding mean and std. The mean and std data for R1 and R2 are saved into all_reward_stat_harmhelp_Rlarge.npy.
The mean and std for R1 and R2 is (-0.94732502, 1.92034349), (-0.01500361, 1.40736504), respectively.