dataset_info:
features:

• name: user_queries
  list: string
• name: fn_schema
  dtype: string
• name: format
  dtype: string
• name: chosen_param
  dtype: string
• name: filename
  dtype: string
  splits:
• name: train
  num_bytes: 595692
  num_examples: 150
  download_size: 306606
  dataset_size: 595692
  configs:
• config_name: default
  data_files:
  • split: train
    path: data/train-*
    task_categories:
• text-generation
  language:
• en
  tags:
• function-calling
• instruction-following
• benchmark
• llm-evaluation

Paper

Instruction-Following Evaluation in Function Calling for Large Language Models

Code

https://github.com/Skripkon/IFEval-FC

IFEval-FC Leaderboard

Benchmark overview

IFEval-FC is similar to IFEval in that it evaluates instruction following, but with an important difference:
it evaluates instruction following specifically in argument values during function calling.

Many formats were taken or adapted from the original IFEval paper, but some new ones were also created.

Every task was synthetically generated via a complex pipeline and further validated (20+ professional annotators were involved).

Sample Usage

Set up and evaluation

1. Prepare an environment

python3.11 -m venv .venv
source .venv/bin/activate

2. Run script to install everything you need

scripts/./install_dependencies.sh

3. Follow the instructions in the welcome message (which will appear after running the installation script above) to reproduce the evaluation.

View results

To enhance your experience, we provide a user-friendly results viewer that lets you easily see which formats your model performs best and worst on.

Simply run the command below and follow the prompt to select a results file.

  python3 scripts/view_results.py

Appendix

Expanded table showing the evaluation of various models, grouped by instruction type.

Citation

@misc{skripko2025instructionfollowingevaluationfunctioncalling,
title={Instruction-Following Evaluation in Function Calling for Large Language Models},
author={Nikolai Skripko},
year={2025},
eprint={2509.18420},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2509.18420},
}