shamim01/Ch3Ef

license: apache-2.0
size_categories:

• n<1K
  language:
• en
  pretty_name: Ch3Ef
  viewer: false

Assessment of Multimodal Large Language Models in Alignment with Human Values

Ch3Ef dataset is a Compreh3ensive Evaluation dataset and strategy for assessing alignment with human expectations.
It contains 1002 human-annotated data samples, covering 12 domains and 46 tasks based on the hhh (helpful, honest, harmless) principle.

🌐 Project Page

🏆 Leaderboard

📄 Paper

Format

{
  "id": "id",
  "query": "question for the image",
  "image": "a list of image path",
  "options": "candidate answers",
  "source": "task name"
}

Examples

Citation

@misc{shi2024assessment,
      title={Assessment of Multimodal Large Language Models in Alignment with Human Values}, 
      author={Zhelun Shi and Zhipin Wang and Hongxing Fan and Zaibin Zhang and Lijun Li and Yongting Zhang and Zhenfei Yin and Lu Sheng and Yu Qiao and Jing Shao},
      year={2024},
      eprint={2403.17830},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}