license: apache-2.0
task_categories:

• text-generation
  language:
• it
  pretty_name: ifeval-ita

Dataset Card for IFEval

Dataset Description

• Repository: https://github.com/google-research/google-research/tree/master/instruction_following_eval
• Paper: https://huggingface.co/papers/2311.07911

Dataset Summary

This dataset contains the prompts used in the Instruction-Following Eval (IFEval) benchmark for large language models. It contains around 500 "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times" which can be verified by heuristics. To load the dataset, run:

from datasets import load_dataset

ifeval = load_dataset("mii-llm/ifeval-ita")

Supported Tasks and Leaderboards

The IFEval dataset is designed for evaluating chat or instruction fine-tuned language models and is one of the core benchmarks used in the Open LLM Leaderboard.

Languages

The data in IFEval are in Italian.

Dataset Structure

Dataset Translation

The dataset was translated using Claude Opus using a draft and refine technique. Only the refined translation was kept.

You are a professional translation system that accurately translates multiple-choice exercises from English to Italian. Follow these steps to ensure high-quality translations:

1. Provide an initial translation within <traduzione></traduzione> tags.
2. Propose corrections, if necessary, within <correzioni></correzioni> tags, always re-reading the input problem.
3. Write the final, polished translation within <traduzione-finale></traduzione-finale> tags.

Adhere to the following requirements:
1. Deliver top-notch, professional translations in Italian.
2. Ensure the translated text is fluent, grammatically perfect, and uses standard Italian without regional bias.
3. Accurately translate mathematical terms, notations, and equations, preserving their original meaning and structure.
4. Focus solely on translating content without providing explanations, adding extra information, or copying the source text verbatim.

Always use the following output format:

<traduzione>
    <domanda>[write the translated question here]</domanda>
    <opzioni>
        <opzione>[write the translated option here]</opzione>
        <opzione>[write the translated option here]</opzione>
        <opzione>[write the translated option here]</opzione>
        ...
    </opzioni>
</traduzione>
<correzioni>
[write your corrections here, analyzing the translation quality, errors, and providing suggestions regarding the exercise and given options]
</correzioni>
<traduzione-finale>
    <domanda>[write the translated question here]</domanda>
    <opzioni>
        <opzione>[write the translated option here]</opzione>
        <opzione>[write the translated option here]</opzione>
        <opzione>[write the translated option here]</opzione>
        ...
    </opzioni>
</traduzione-finale>

From now on, only write in Italian and translate all incoming messages. Ensure the best translation possible.

Data Instances

An example of the train split looks as follows:

{"key": 1000, "prompt": "Scrivi un riassunto di almeno 300 parole della pagina Wikipedia \"https://en.wikipedia.org/wiki/Raymond_III,_Count_of_Tripoli\". Non utilizzare virgole e evidenzia almeno 3 sezioni che hanno titoli in formato markdown come ad esempio *sezione evidenziata parte 1* *sezione evidenziata parte 2* *sezione evidenziata parte 3*.", "instruction_id_list": ["punctuation:no_comma", "detectable_format:number_highlighted_sections", "length_constraints:number_words"], "kwargs": [{}, {"num_highlights": 3}, {"relation": "at least", "num_words": 300}]}
{"key": 1001, "prompt": "Sto pianificando un viaggio in Giappone e gradirei che tu componessi un itinerario del mio viaggio in stile shakespeariano. Non \u00e8 consentito l'uso di virgole nella risposta.", "instruction_id_list": ["punctuation:no_comma"], "kwargs": [{}]}

Data Fields

The data fields are as follows:

• key: A unique ID for the prompt.
• prompt: Describes the task the model should perform.
• instruction_id_list: An array of verifiable instructions. See Table 1 of the paper for the full set with their descriptions.
• kwargs: An array of arguments used to specify each verifiable instruction in instruction_id_list.

Data Splits

Licensing Information

The dataset is available under the Apache 2.0 license.

Citation Information

@misc{zhou2023instructionfollowingevaluationlargelanguage,
      title={Instruction-Following Evaluation for Large Language Models}, 
      author={Jeffrey Zhou and Tianjian Lu and Swaroop Mishra and Siddhartha Brahma and Sujoy Basu and Yi Luan and Denny Zhou and Le Hou},
      year={2023},
      eprint={2311.07911},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2311.07911}, 
}