dataset_info:
features:

• name: row_id
  dtype: string
• name: messages
  list:
  • name: role
    dtype: string
  • name: content
    dtype: string
• name: instruction_id_list
  sequence: string
• name: kwargs
  list:
  • name: capital_frequency
    dtype: int64
  • name: capital_relation
    dtype: string
  • name: end_phrase
    dtype: string
  • name: first_word
    dtype: string
  • name: forbidden_words
    sequence: string
  • name: frequency
    dtype: int64
  • name: keyword
    dtype: string
  • name: keywords
    sequence: string
  • name: let_frequency
    dtype: int64
  • name: let_relation
    dtype: string
  • name: letter
    dtype: string
  • name: nth_paragraph
    dtype: int64
  • name: num_bullets
    dtype: int64
  • name: num_highlights
    dtype: int64
  • name: num_paragraphs
    dtype: int64
  • name: num_placeholders
    dtype: int64
  • name: num_sections
    dtype: int64
  • name: num_sentences
    dtype: int64
  • name: num_words
    dtype: int64
  • name: postscript_marker
    dtype: string
  • name: relation
    dtype: string
  • name: section_spliter
    dtype: string
    splits:
• name: train
  num_bytes: 31000000
  num_examples: 10070
  download_size: 31000000
  dataset_size: 31000000
  configs:
• config_name: default
  data_files:
  • split: train
    path: kimify-ifeval-like.jsonl
    language:
• en
  license: mit
  task_categories:
• text-generation
  tags:
• instruction-following
• ifeval
• constraint-satisfaction
• verified
  pretty_name: Kimify IFEval-Like Dataset
  size_categories:
• 10K<n<100K

Kimify IFEval-Like Dataset

Dataset Description

This dataset contains 10,070 verified instruction-following conversations in the IFEval format. Each example includes:

• A user prompt with embedded constraints
• An assistant response that satisfies those constraints
• Metadata describing the constraint types and parameters

All examples have been programmatically verified using the instruction-following-eval library (based on Google Research's IFEval) to ensure 100% constraint satisfaction.

The repository also includes the verify_dataset.py script that verifies the constraints. You can use the user prompts in this dataset to generate responses with any model, save the result into a similarly-formatted JSONL dataset, then use this script to ensure the responses actually match the prompts. Note that most user prompts in the dataset are used twice.

Dataset Summary

• Total Examples: 10,070
• Format: Conversational (user-assistant pairs)
• Verification: 100% pass rate using official IFEval checkers
• Languages: English
• Constraint Types: 28+ different instruction-following constraints

Supported Tasks

• Instruction following
• Constrained text generation
• Multi-constraint satisfaction
• Fine-tuning language models for precise instruction adherence

Dataset Structure

Data Instances

Each example contains:

{
  "row_id": "kimify-ifeval-like_1",
  "messages": [
    {
      "role": "user",
      "content": "Your answer must contain exactly 3 bullet points..."
    },
    {
      "role": "assistant",
      "content": "* First point...\n* Second point...\n* Third point..."
    }
  ],
  "instruction_id_list": [
    "detectable_format:number_bullet_lists",
    "detectable_content:number_placeholders",
    "keywords:frequency",
    "change_case:english_capital"
  ],
  "kwargs": [
    {
      "num_bullets": 3
    },
    {
      "num_placeholders": 2
    },
    {
      "frequency": 3,
      "keyword": "keyword",
      "relation": "at least"
    },
    {}
  ]
}

Data Fields

• row_id (string): Unique identifier for each example
• messages (list): Conversation messages with role and content
  • role (string): Either "user" or "assistant"
  • content (string): Message text
• instruction_id_list (list of strings): Constraint types applied to this example
• kwargs (list of dicts): Parameters for each constraint (one dict per constraint in instruction_id_list)
  • Each dict contains only the relevant parameters for that specific constraint
  • Unused parameters are omitted (not set to null)
  • Empty dict {} indicates a constraint with no parameters
  • Common parameters: num_bullets, frequency, keyword, num_placeholders, relation, etc.

Data Splits

The dataset is provided as a single split with 10,070 examples.

Constraint Types

The dataset includes the following constraint categories:

• Format constraints: bullet lists, numbered sections, titles, JSON format, highlights
• Length constraints: word count, sentence count, paragraph count
• Content constraints: placeholders, postscripts, specific keywords
• Case constraints: all uppercase, all lowercase, title case
• Letter frequency: specific letter occurrence requirements
• Forbidden words: words that must not appear
• Start/end constraints: specific opening or closing phrases
• Combination constraints: multiple simultaneous requirements

See the instruction-following-eval documentation and Google Research's IFEval for detailed descriptions of each constraint type.

Dataset Creation

The user prompts and constraints were sourced from the HuggingFace H4 ifeval-like-data dataset. Unfortunately some of the prompts in the dataset were contaminated with answers; we used Qwen-235B-A22B-Instruct-2507 to "clean" these prompts.

We generated the assistant responses using Kimi K2 0905[https://huggingface.co/moonshotai/Kimi-K2-Instruct-0905], with a system prompt encouraging concise instruction following. Then we verified the responses against the constraints and dropped the lines that failed the verification (the pass rate was above 90%).

Verification Script

To verify that entries in this dataset, or a similar dataset generated from the same user prompts with a model of your choice, use the provided script:

# Install dependencies
pip install -r requirements.txt

# Run verification
python verify_dataset.py kimify-ifeval-like.jsonl

This will check all 10,070 examples and confirm 100% pass rate.

Known Limitations

• All examples are in English
• Constraints are verified deterministically; we did not verify the writing quality or factual accuracy

Acknowledgments

This dataset uses the instruction-following-eval library for verification, which is based on Google Research's IFEval benchmark:

• Original IFEval by Google Research: https://github.com/google-research/google-research/tree/master/instruction_following_eval
• Copyright 2024 The Google Research Authors
• Licensed under the Apache License, Version 2.0

The verification library is maintained by Jose Javier Gonzalez Ortiz and is also licensed under Apache 2.0.

The user prompts and constraints are sourced from the HuggingFace H4 ifeval-like-data dataset that is published by the HuggingFace H4 team.

Claude Code was used to create the service scripts, including the verification script in this repository. No data from Claude is present in the dataset itself.