license: mit
language:
- en
Introduction
- Paper: Empowering Reliable Visual-Centric Instruction Following in MLLMs
- Github: KerenWLHe/VC-IFEval
😊 This is the official repo of VC-IFEngine datasets.
📊 Image sources: LLaVA-Instruct, Visual Genome, ALLaVA-4V
Format
SFT — messages format (role: user / assistant):
{
"messages": [
{"role": "user", "content": "<image>\n{instruction}"},
{"role": "assistant", "content": "{response}"}
],
"images": ["Data/CC3M/images/example.jpg"]
}
DPO — conversations + chosen / rejected format:
{
"conversations": [
{"from": "human", "value": "<image>\n{instruction}"}
],
"chosen": {"from": "gpt", "value": "{full-constraint response}"},
"rejected": {"from": "gpt", "value": "{constraint-degraded response}"},
"images": ["Data/CC3M/images/example.jpg"]
}
Citation
@misc{he2026empoweringreliablevisualcentricinstruction,
title={Empowering Reliable Visual-Centric Instruction Following in MLLMs},
author={Weilei He and Feng Ju and Zhiyuan Fan and Rui Min and Minhao Cheng and Yi R. Fung},
year={2026},
eprint={2601.03198},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2601.03198},
}