license: apache-2.0
task_categories:
- text-generation
language: - en
tags: - reward-model
- preference
- evaluation
- benchmark
- best-of-n
pretty_name: RMB Best-of-N
size_categories: - 1K<n<10K
source_datasets: - Zhou-Zoey/RMB-Reward-Model-Benchmark
RMB-BoN
Flattened Best-of-N split of the RMB (Reward Model Benchmark) dataset from Zhou-Zoey/RMB-Reward-Model-Benchmark.
RMB is a comprehensive reward model benchmark covering 49 real-world scenarios across two alignment goals (Helpfulness and Harmlessness), introduced in the ICLR 2025 paper.
Schema
| Column | Type | Description |
|---|---|---|
bon_uid |
str | Unique identifier |
conversation |
list[dict] | Multi-turn conversation context (role, content, language) |
chosen |
str | Best response (bon_best) |
rejected |
list[str] | Loser responses (2-6 per example) |
chosen_model |
str | Model that generated the best response |
rejected_models |
list[str] | Models that generated the loser responses |
alignment_goal |
str | "Helpfulness" or "Harmlessness" |
category |
str | Top-level category |
subcategory |
str | Fine-grained subcategory |
category_path |
str | Full path (e.g., "BoN_set/Helpfulness/Code/Data Management") |
Stats
- 3,786 Best-of-N examples
- 2-6 rejected responses per example
- 2 alignment goals: Helpfulness, Harmlessness
- 49 scenarios
Citation
@inproceedings{zhou2025rmb,
title={RMB: Comprehensively Benchmarking Reward Models in LLM Alignment},
author={Zhou, Zihuiwen and Liu, Hanming and Zhao, Pengfei and Du, Yuanshun and Chen, Hua and Li, Kai and Wang, Yanbo and Sun, Raphael Hongyu and Chen, Liang and Chen, Jingjing},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://arxiv.org/abs/2410.09893}
}
Source
Original data from https://github.com/Zhou-Zoey/RMB-Reward-Model-Benchmark.