language:

• en
  license: odc-by
  size_categories:
• 100K<n<1M
  task_categories:
• text-generation
• reinforcement-learning
  pretty_name: Unified IF VERL Dataset
  tags:
• instruction-following
• evaluation
• ifeval
• reasoning
• reinforcement-learning
• verl
• deduplication
  configs:
• config_name: default
  data_files:
  • split: ifbench_verl
    path: data/ifbench-verl.parquet
  • split: ifeval_rlvr_verl
    path: data/ifeval-rlvr-verl.parquet

Unified IF VERL Dataset

A unified collection of 2 high-quality instruction-following (IF) evaluation datasets in VERL format, deduplicated and optimized for reinforcement learning training.

Dataset Summary

This dataset combines 2 instruction-following evaluation datasets into a single unified collection:

• Total Problems: 110,342 unique problems (after 0.00% deduplication)
• Original Size: 110,346 problems (before deduplication)
• Constraint Types: 79 unique constraint types (25 + 54 from sources)
• Format: VERL (Volcano Engine Reinforcement Learning)
• Language: English
• License: ODC-BY (Open Data Commons Attribution License)

Dataset Structure

Splits

The dataset is organized into 2 named splits, one for each source dataset:

Usage

Load specific split:

from datasets import load_dataset

# Load only one dataset
dataset = load_dataset("sungyub/if-verl-unified", split="ifeval-rlvr-verl")

# Load all datasets
dataset = load_dataset("sungyub/if-verl-unified")

Using with IFEval Scorer:

from datatrove.utils.reward_score import compute_score

# Get an example
example = dataset[0]

# Generate a response
response = "<think>Analysis here</think>\nFinal answer"

# Compute score
score = compute_score(
    data_source="sungyub/if-verl-unified",
    solution_str=response,
    ground_truth=example["reward_model"]["ground_truth"],
    format_type="auto"  # Supports both XML and GPT OSS formats
)

Data Format

All splits follow the VERL (Volcano Engine Reinforcement Learning) format:

{
    "data_source": str,        # Dataset identifier
    "prompt": [                # Chat template format
        {
            "role": "user",
            "content": "instruction with constraints"
        }
    ],
    "ability": "instruction_following",
    "reward_model": {          # Verification info
        "style": "ifeval",
        "ground_truth": str     # Python literal string with constraint specs
    },
    "extra_info": {            # Metadata
        "index": int,
        "split": str,
        "original_dataset": str  # Source dataset name
    },
    "dataset": "ifeval"
}

Deduplication Process

The dataset underwent a rigorous 2-phase deduplication process:

Phase 1: Intra-dataset deduplication

• Removed duplicates within each dataset
• Reduced 110,346 → 110,342 problems (0.00% reduction)

Phase 2: Inter-dataset deduplication

• Removed duplicates across datasets using size-based priority
• Priority: smallest datasets first (preserves rare problems)
• Reduced 110,342 → 110,342 problems (0.00% reduction)

Overall: 0.00% duplicate removal

Deduplication Method

• SHA-256 hash-based exact matching
• Conservative text normalization (preserves formatting)
• VERL format validation for all outputs
• Size-based priority (smallest datasets preserved first)

Source Datasets

ifeval-rlvr-verl (14,970 problems) - ODC-BY

• Source: allenai/RLVR-IFeval
• 25 distinct constraint types
• Instruction-following evaluation from Allen Institute

ifbench-verl (95,372 problems) - ODC-BY

• 54 distinct constraint types
• Comprehensive instruction-following benchmark
• Multi-source aggregation (95K examples)

Constraint Types Coverage

The unified dataset includes 79 total constraint types across 9 categories:

• Keywords (4 types): existence, frequency, forbidden words, letter frequency
• Language (1 type): response language requirements
• Length Constraints (4 types): paragraphs, words, sentences, nth paragraph
• Detectable Content (2 types): postscript, placeholders
• Detectable Format (6 types): bullet lists, title, constrained response, highlighted sections, sections, JSON
• Combination (2 types): repeat prompt, two responses
• Case Changes (3 types): uppercase, lowercase, capital word frequency
• Start/End (2 types): end checker, quotation
• Punctuation (1 type): no comma

Use Cases

Reinforcement Learning Training:

• Post-training for instruction-following capabilities
• Constraint satisfaction learning
• Reward modeling with verifiable constraints

Evaluation:

• Systematic instruction-following evaluation
• Constraint compliance testing
• Multi-constraint scenarios

Fine-tuning:

• Improving instruction adherence
• Constraint-aware generation
• Format compliance training

Dataset Creation

Deduplication Pipeline

1. Phase 1 (Intra-dataset):

   • Process each dataset independently
   • Remove exact duplicates based on normalized instruction text
   • Validate VERL format

2. Phase 2 (Inter-dataset):

   • Process datasets in size-based priority order (smallest first)
   • Remove duplicates across datasets
   • Add original_dataset field for tracking

Priority Rationale

Size-based priority (smallest datasets first) was chosen to:

• Preserve rare constraint types from smaller datasets
• Maximize constraint type diversity
• Retain unique instruction patterns

Limitations

• Some high-frequency constraints may be overrepresented
• English language only
• Constraint specifications in Python literal format
• Some overlap expected between IF evaluation datasets

License and Attribution

License: ODC-BY (Open Data Commons Attribution License)

Under ODC-BY, you are free to:

• Share: Copy and redistribute the data
• Adapt: Transform and build upon the data
• Use commercially: Use the data for commercial purposes

Requirements:

• Attribution: You must give appropriate credit to original data sources
• Indicate if changes were made

Source Attributions:

• ifeval-rlvr-verl: allenai/RLVR-IFeval (ODC-BY)
• ifbench-verl: Multiple sources, aggregated (ODC-BY)

Citation

If you use this dataset, please cite the original source datasets and this unified collection:

@dataset{if-verl-unified,
  title={Unified IF VERL Dataset},
  author={Sungyub Kim},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/datasets/sungyub/if-verl-unified}
}

@misc{rlvr-ifeval,
  title={RLVR-IFeval: Instruction Following Evaluation Dataset},
  author={Allen Institute for AI},
  year={2024},
  url={https://huggingface.co/datasets/allenai/RLVR-IFeval}
}

@misc{zhou2023instructionfollowing,
  title={Instruction-Following Evaluation for Large Language Models},
  author={Jeffrey Zhou and Tianjian Lu and Swaroop Mishra and Siddhartha Brahma and Sujoy Basu and Yi Luan and Denny Zhou and Le Hou},
  year={2023},
  eprint={2311.07911},
  archivePrefix={arXiv}
}

Dataset Card Authors

• Sungyub Kim (sungyub)