language:

• en
• ru
  license: apache-2.0

IFEval - Instruction Following Evaluation Dataset

This dataset is designed for evaluating how well language models follow specific instructions when generating responses. It serves as the default evaluation data for IFEval repository.

Dataset Description

The IFEval dataset contains prompts with specific instructions designed to test language models' ability to follow directions precisely. It is intended for use with the evaluation framework described in the paper Instruction-Following Evaluation for Large Language Models.

The dataset is available in two language variants:

• English (en)
• Russian (ru)

Both variants are equivalent in terms of the instructions and evaluation criteria, allowing for cross-lingual comparison of instruction-following capabilities.

Usage

This dataset is primarily intended to be used with the IFEval repository. The framework automatically loads the appropriate dataset based on the language setting when no specific input data is provided.

Dataset Structure

Each example in the dataset contains:

• key: A unique identifier for the example
• prompt: The input text given to the model
• instruction_id_list: List of instruction IDs being tested
• kwargs: Parameters for each instruction

Citation

If you use this dataset in your research, please cite:

@article{zhou2023instruction,
  title={Instruction-Following Evaluation for Large Language Models},
  author={Zhou, Jeffrey and Lu, Tianjian and Mishra, Swaroop and Brahma, Siddhartha and Basu, Sujoy and Luan, Yi and Zhou, Denny and Hou, Le},
  journal={arXiv preprint arXiv:2311.07911},
  year={2023}
}