pretty_name: Uni-SafeBench
language:

• en
  license: cc-by-nc-4.0
  task_categories:
• text-generation
• visual-question-answering
• image-to-text
  tags:
• multimodal
• safety
• unified-multimodal-models
• harmful-content
• benchmark
  size_categories:
• 1K<n<10K
  gated: true
  extra_gated_prompt: >-
  Uni-SafeBench contains harmful, and unsafe textual and visual
  content. Access is restricted to academic and non-commercial safety research.
  By requesting access, you agree to use this dataset only for research purposes
  and not to use it for generating harmful content, improving harmful instruction
  following, bypassing safety mechanisms, or redistributing unsafe content.
  extra_gated_fields:
  Full name: text
  Affiliation: text
  Institutional email: text
  Intended use:
  type: select
  options:
  - Academic research
  - Non-commercial safety evaluation
  - Education
  - Other
  
  Research purpose: text
  I agree to use this dataset only for non-commercial academic research: checkbox
  I agree not to use this dataset to generate harmful content: checkbox
  I agree not to use this dataset to bypass or weaken safety mechanisms: checkbox
  I agree not to redistribute the dataset or its unsafe samples to unauthorized users: checkbox
  configs:
• config_name: t2i
  data_files:
  • split: test
    path: data/t2i/test-*.parquet
• config_name: text_generation
  data_files:
  • split: test
    path: data/text_generation/test-*.parquet
• config_name: text_guided_image_editing
  data_files:
  • split: test
    path: data/text_guided_image_editing/test-*.parquet
• config_name: vqa_safety_i_safety_t
  data_files:
  • split: test
    path: data/vqa_safety_i_safety_t/test-*.parquet
• config_name: vqa_safety_i_unsafety_t
  data_files:
  • split: test
    path: data/vqa_safety_i_unsafety_t/test-*.parquet
• config_name: vqa_unsafety_i_safety_t
  data_files:
  • split: test
    path: data/vqa_unsafety_i_safety_t/test-*.parquet
• config_name: vqa_unsafety_i_unsafety_t
  data_files:
  • split: test
    path: data/vqa_unsafety_i_unsafety_t/test-*.parquet

Uni-SafeBench

Content Warning

Uni-SafeBench contains harmful, and unsafe textual and visual content. The dataset is intended only for academic and non-commercial research on the safety evaluation of unified multimodal large models.

The dataset should not be used to generate harmful content, improve harmful instruction following, bypass safety mechanisms, or deploy unsafe generation systems.

Dataset Description

Uni-SafeBench is a safety evaluation benchmark for Unified Multimodal Large Models. It is designed to evaluate safety risks across both multimodal understanding and generation scenarios, including text generation, text-to-image generation, text-guided image editing, and visual question answering.

The benchmark focuses on evaluating whether unified multimodal models can safely handle harmful multimodal inputs and whether unification introduces additional safety risks across different input-output modalities.

Intended Use

Uni-SafeBench is intended for:

• academic research on multimodal safety;
• non-commercial safety evaluation of multimodal large models;
• benchmarking model robustness against harmful multimodal instructions;
• developing safer evaluation protocols and mitigation methods.

Prohibited Use

Users should not use this dataset to:

• generate harmful content;
• train or improve models to follow harmful instructions;
• bypass, weaken, or attack model safety mechanisms;
• deploy harmful or unsafe generation systems;
• redistribute the dataset or its unsafe samples to unauthorized users;
• use the dataset for commercial purposes without permission.

Access and Use Agreement

By requesting access to Uni-SafeBench, users agree that:

1. The dataset will be used only for academic or non-commercial safety research.
2. The dataset will not be used to train, fine-tune, or improve models for generating harmful content.
3. The dataset will not be used to bypass, weaken, or attack safety mechanisms.
4. The dataset or its unsafe samples will not be redistributed to unauthorized users.
5. Any publication or derivative work using this dataset should cite Uni-SafeBench.
6. The dataset authors reserve the right to revoke access if misuse is identified.

Dataset Configurations

The dataset contains the following configurations:

• t2i
• text_generation
• text_guided_image_editing
• vqa_safety_i_safety_t
• vqa_safety_i_unsafety_t
• vqa_unsafety_i_safety_t
• vqa_unsafety_i_unsafety_t

Loading the Dataset

After access is granted, the dataset can be loaded with the Hugging Face datasets library.

from datasets import load_dataset

ds = load_dataset(
    "Hades2002/Uni-SafeBench",
    "text_generation",
    split="test",
    token=True,
)

print(ds[0])

For image-based configurations:

from datasets import load_dataset

ds = load_dataset(
    "Hades2002/Uni-SafeBench",
    "vqa_safety_i_safety_t",
    split="test",
    token=True,
)

sample = ds[0]
print(sample)
sample["image"].show()

Available configurations include:

configs = [
    "t2i",
    "text_generation",
    "text_guided_image_editing",
    "vqa_safety_i_safety_t",
    "vqa_safety_i_unsafety_t",
    "vqa_unsafety_i_safety_t",
    "vqa_unsafety_i_unsafety_t",
]

License

This dataset is released under the CC BY-NC 4.0 license. It is intended for non-commercial research use only.

Citation

@article{peng2026does,
  title={Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Models},
  author={Peng, Zixiang and Xu, Yongxiu and Zhang, Qinyi and Shen, Jiexun and Zhang, Yifan and Xu, Hongbo and Wang, Yubin and Gou, Gaopeng},
  journal={arXiv preprint arXiv:2604.00547},
  year={2026}
}