license: apache-2.0
task_categories:

• reinforcement-learning
• text-generation
  language:
• en
  tags:
• instruction-following
• rlvr
• grpo
• ifeval
• ifbench
• lexinstructeval
  size_categories:
• 10K<n<100K
  configs:
• config_name: default
  data_files: "data/**/*.parquet"
  default: true
• config_name: IFEval
  data_files: "data/IFEval/*.parquet"
• config_name: IFBench
  data_files: "data/IFBench/*.parquet"
• config_name: LexInstructEval
  data_files: "data/LexInstructEval/*.parquet"

RL-Instruction-Following-Dataset 🎯

Overview

This dataset is designed to enhance the Instruction Following capabilities of Large Language Models (LLMs) through Reinforcement Learning (RL). Unlike subjective preference datasets (e.g., standard RLHF), this dataset focuses on Objective, Rule-Based Constraints.

Each entry provides a prompt with specific hard constraints (format, length, lexical keywords) and a structured ground_truth object used for programmatic reward computation.

Dataset Structure & Subsets

The dataset contains 3 distinct subsets, each targeting a different dimension of instruction following:

Data Fields

• data_source: Origin identifier (e.g., ifeval_llm).
• prompt: The user instruction containing the constraint.
• ability: The specific domain (ifeval, lexical, ifbench).
• reward_model: A JSON object containing the ground truth for reward calculation.
  • style: rule
  • ground_truth: Dictionary with kwargs and instruction_id_list required for verification.

Example Entry

{
  "ability": "ifeval",
  "prompt": [
    {"role": "user", "content": "Write a story... Your answer must contain exactly 4 bullet points."}
  ],
  "reward_model": {
    "style": "rule",
    "ground_truth": "{\\"meta_info\\": {\\"kwargs\\": [{\\"num_bullets\\": 4}], \\"instruction_id_list\\": [\\"detectable_format:number_bullet_lists\\"]}}"
  }
}

Construction & Attribution

To ensure high quality and diversity, this dataset was constructed using a hybrid pipeline:

1. Base Queries (Source):
   The underlying semantic instructions (topics and questions) are sourced from high-quality open-source instruction datasets, primarily:

   • Infinity Instruct
   • AM-think (and related open-source SFT datasets).
   • Note: These queries serve as the "carrier" for the constraints. We selected queries that naturally allow for complex formatting or logical modifications.

2. Instruction Synthesis:
   Specific rule-based constraints (e.g., "use 4 bullet points", "include the word 'apple' twice") were synthesized and programmatically injected into the base queries to create the final prompts.

3. Reward Logic:
   The ground_truth and validation logic strictly align with the official evaluation protocols of IFEval, IFBench, and LexInstructEval.

⚠️ Note on Decontamination: This dataset contains synthetic instructions derived from training sets or general open queries. It does NOT contain the exact test samples from the original benchmarks to avoid data leakage during evaluation.

Usage

Loading the Dataset

from datasets import load_dataset

# Load the full mixed dataset (includes all subsets)
ds_all = load_dataset("your-username/RL-Instruction-Following-Dataset", "default")

# Load a specific subset (e.g., IFEval) for targeted training
ds_ifeval = load_dataset("your-username/RL-Instruction-Following-Dataset", "IFEval")
ds_lexical = load_dataset("your-username/RL-Instruction-Following-Dataset", "LexInstructEval")

License

This dataset is released under the Apache 2.0 license.