license: apache-2.0
language:
- en
size_categories: - 10K<n<100K
argilla/distilabel-intel-orca-dpo-pairs in ChatML format, ready to use in HuggingFace TRL's DPO Trainer.
Python code used for conversion:
from datasets import load_dataset
dataset = load_dataset("argilla/distilabel-intel-orca-dpo-pairs", split="train")
def format(columns):
prompt = f"<|im_start|>user\n{columns['input']}<|im_end|>\n<|im_start|>assistant\n"
if (columns['system']):
prompt = f"<|im_start|>system\n{columns['system']}<|im_end|>\n{prompt}"
return {
"prompt": prompt,
"chosen": f"{columns['chosen']}<|im_end|>",
"rejected": f"{columns['rejected']}<|im_end|>",
}
dataset.map(format).select_columns(['prompt', 'chosen', 'rejected', 'status', 'chosen_score', 'in_gsm8k_train']).to_parquet("train.parquet")