Dataset Card for CardinalPrefs

CardinalPrefs is a dataset of 25K pieces of cardinal human feedback used in the paper Beyond Ordinal Preferences: Why Alignment Needs Cardinal Human Feedback. We gather (prompt, response 1, response 2) tuples from the HH-RLHF and LMSys chat datasets.

We include a full description of the dataset, including how we sourced human participants and our elicitation strategy in the paper.

Dataset Structure

Each row in the dataset is labeled with:

• prompt: The initial user prompt to start the conversation.
• response1 and response2: The two full conversation transcripts. Only the last assistant response differs between these two transcripts.
• net_utility: The participant's willingness-to-pay (how many $ the participant is willing to pay) to upgrade response 2 -> response 1. In other words, net_utility = u(response1) - u(response2). When negative this means that response 2 is preferred to response 1.
• data_source: Which dataset this (prompt, response 1, response 2) tuple is source from.
• labeler: ID for the human labeler who provided this piece of feedback. We paid 4 human labelers, with 2 of them providing the great majority of the data.
• is_multi_turn

Citation

BibTeX:

@misc{whitfill2025ordinalpreferencesalignmentneeds,
      title={Beyond Ordinal Preferences: Why Alignment Needs Cardinal Human Feedback}, 
      author={Parker Whitfill and Stewy Slocum},
      year={2025},
      eprint={2508.08486},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2508.08486}, 
}