pretty_name: Dataset Featurization
language:

• en
  license:
• mit
  task_categories:
• feature-extraction
  task_ids:
• language-modeling
  configs:
• config_name: hh-rlhf
  data_files:
  • split: train
    path: data/hh-rlhf/train.csv
  • split: test
    path: data/hh-rlhf/test.csv
• config_name: hh-rlhf-featurization
  data_files:
  • split: train
    path: data/hh-rlhf-featurization/train.csv
• config_name: hh-rlhf-featurization-evaluation
  data_files:
  • split: train
    path: data/hh-rlhf-featurization-evaluation/train.csv
• config_name: hh-rlhf-bon
  data_files:
  • split: train
    path: data/hh-rlhf/bon.csv
• config_name: shp
  data_files:
  • split: train
    path: data/shp/train.csv
  • split: test
    path: data/shp/test.csv
• config_name: shp-featurization
  data_files:
  • split: train
    path: data/shp-featurization/train.csv
• config_name: shp-featurization-evaluation
  data_files:
  • split: train
    path: data/shp-featurization-evaluation/train.csv
• config_name: shp-bon
  data_files:
  • split: train
    path: data/shp/bon.csv

Dataset Featurization: Compositional Preference Modeling

This repository contains the datasets used in our case study on compositional preference modeling from Dataset Featurization, demonstrating how our unsupervised featurization pipeline can produce features describing human preferences and match expert-level produced features. This case study is built on top of Compositional Preference Modeling (CPM).

HH-RLHF - Featurization

Utilizing HH-RLHF dataset, we provide access to the input dataset and the evaluation stage outputs containing candidate features and their binary valuations:

import datasets
text_df = load_dataset("Bravansky/compositional-preference-modeling", "hh-rlhf-featurization", split="train").to_pandas()
evaluation_df = load_dataset("Bravansky/compositional-preference-modeling", "hh-rlhf-featurization-evaluation", split="train").to_pandas()

SHP - Featurization

Similarly, we provide access to the input dataset from SHP and the evaluation stage outputs containing candidate features and their binary valuations:

import datasets
text_df = load_dataset("Bravansky/compositional-preference-modeling", "shp-featurization", split="train").to_pandas()
evaluation_df = load_dataset("Bravansky/compositional-preference-modeling", "shp-featurization-evaluation", split="train").to_pandas()

HH-RLHF - Experiments

For our experiments following the CPM methodology, we provide two HH-RLHF datasets (train and test) with our assigned features:

import datasets
train_df = load_dataset("Bravansky/compositional-preference-modeling", "hh-rlhf", split="train").to_pandas()
test_df = load_dataset("Bravansky/compositional-preference-modeling", "hh-rlhf", split="test").to_pandas()

SHP - Experiments

Similarly, we provide train and test datasets for evaluating CPM performance on SHP with our assigned features:

import datasets
train_df = load_dataset("Bravansky/compositional-preference-modeling", "shp", split="train").to_pandas()
test_df = load_dataset("Bravansky/compositional-preference-modeling", "shp", split="test").to_pandas()

HH-RLHF - Best of N (BoN)

Lastly, we evaluate our HH-RLHF features over the BoN dataset constructed with a subset of HH-RLHF prompts and Flan T5-large generations and provide the results:

import datasets
bon_df = load_dataset("Bravansky/compositional-preference-modeling", "hh-rlhf-bon", split="train").to_pandas()

SHP - Best of N (BoN)

We do the same for SHP features and provide the BoN dataset constructed with a subset of SHP prompts and Flan T5-large generations and provide the results:

import datasets
bon_df = load_dataset("Bravansky/compositional-preference-modeling", "shp-bon", split="train").to_pandas()