license: apache-2.0
task_categories:

• text-generation
  language:
• en
  size_categories:
• 1K<n<10K

license: apache-2.0
task_categories:

• text-generation
• evaluation
  language:
• en

Dataset Summary

IFEval++ is an evaluation dataset introduced in the paper “Revisiting the Reliability of Language Models in Instruction-Following”.
It is designed to assess instruction-following reliability and robustness in large language models.
IFEval++ extends the original IFEval benchmark with systematic cleaning, augmentation, and validation.

IFEval++ contains both the original IFEval examples and augmented examples for more comprehensive evaluation.

Dataset Structure

Data Instances

An example of IFEval++ looks as follows:

{
    "key": "1000:ct_alteration:1",
    "prompt": "Write a 350+ word summary of the wikipedia page "https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli". Do not use any commas and highlight at least 4 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*, *highlighted section part 4*.",
    "instruction_id_list": [
        "punctuation:no_comma",
        "detectable_format:number_highlighted_sections",
        "length_constraints:number_words"
    ],
    "kwargs": [
        {

        },
        {
            "num_highlights": 4
        },
        {
            "relation": "at least",
            "num_words": 350
        }
    ],
    "original_prompt": "Write a 300+ word summary of the wikipedia page "https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli". Do not use any commas and highlight at least 3 sections that has titles in markdown format, for example *highlighted section part 1*, *highlighted section part 2*, *highlighted section part 3*.",
    "original_instruction_id_list": [
        "punctuation:no_comma",
        "detectable_format:number_highlighted_sections",
        "length_constraints:number_words"
    ],
    "original_kwargs": [
        {

        },
        {
            "num_highlights": 3
        },
        {
            "relation": "at least",
            "num_words": 300
        }
    ]
}

Usage

To use the data, you can load it using the following code:

from datasets import load_dataset

dataset = load_dataset("RebeccaYU920/ifeval-pp")

Citation

You can cite our paper or this dataset:

@misc{dong2025revisitingreliabilitylanguagemodels,
      title={Revisiting the Reliability of Language Models in Instruction-Following}, 
      author={Jianshuo Dong and Yutong Zhang and Yan Liu and Zhenyu Zhong and Tao Wei and Chao Zhang and Han Qiu},
      year={2025},
      eprint={2512.14754},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/2512.14754}, 
}