license: apache-2.0
task_categories:
- text-generation
language: - en
tags: - reward-model
- preference
- evaluation
- benchmark
pretty_name: RMB Pairwise
size_categories: - 10K<n<50K
source_datasets: - Zhou-Zoey/RMB-Reward-Model-Benchmark
RMB-Pairwise
Flattened pairwise split of the RMB (Reward Model Benchmark) dataset from Zhou-Zoey/RMB-Reward-Model-Benchmark.
RMB is a comprehensive reward model benchmark covering 49 real-world scenarios across two alignment goals (Helpfulness and Harmlessness), introduced in the ICLR 2025 paper.
Schema
| Column | Type | Description |
|---|---|---|
pair_uid |
str | Unique pair identifier |
conversation |
list[dict] | Multi-turn conversation context (role, content, language) |
chosen |
str | Preferred response |
rejected |
str | Non-preferred response |
chosen_model |
str | Model that generated the chosen response |
rejected_model |
str | Model that generated the rejected response |
alignment_goal |
str | "Helpfulness" or "Harmlessness" |
category |
str | Top-level category (e.g., "Code", "Chat", "Reasoning", "S1"-"S11") |
subcategory |
str | Fine-grained subcategory |
category_path |
str | Full path (e.g., "Pairwise_set/Helpfulness/Code/Data Management") |
Stats
- 17,131 pairwise examples
- 2 alignment goals: Helpfulness, Harmlessness
- 49 scenarios
Citation
@inproceedings{zhou2025rmb,
title={RMB: Comprehensively Benchmarking Reward Models in LLM Alignment},
author={Zhou, Zihuiwen and Liu, Hanming and Zhao, Pengfei and Du, Yuanshun and Chen, Hua and Li, Kai and Wang, Yanbo and Sun, Raphael Hongyu and Chen, Liang and Chen, Jingjing},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://arxiv.org/abs/2410.09893}
}
Source
Original data from https://github.com/Zhou-Zoey/RMB-Reward-Model-Benchmark.