pretty_name: StructAttack Dataset
language:

• en
  license: other
  task_categories:
• visual-question-answering
  tags:
• ai-safety
• red-teaming
• jailbreak
• multimodal
  size_categories:
• 1K<n<10K

StructAttack Dataset

Prepared research inputs for Models as Lego Builders: Assembling Malice from Benign Blocks via
Semantic Blueprints (CVPR 2026). The corresponding source code is available at
https://github.com/Yef23/StructAttack.

Content warning and intended use

This dataset contains unsafe instructions and adversarial text-image inputs. It is intended for
authorized AI-safety research, model evaluation, and reproducibility. Do not use it to facilitate
harm, target real systems without permission, or deploy unsafe generated outputs. Users are
responsible for following applicable laws, provider policies, and upstream dataset licenses.

Contents

Every JSONL record follows the same schema and points to a content-addressed image within its
dataset directory. The prepared release contains 1,944 AdvBench-M records and 2,450 SafeBench
records across method-specific manifests. Images shared by multiple records are stored only once.

SafeBench JOOD inputs are intentionally absent. The historical experiment archive contains model
results but not the exact corresponding images or generation script, so this release does not
reconstruct or misrepresent them. SafeBench categories 8, 9, and 10 from the upstream 500-record
file are excluded; the released manifests contain the 350 samples used in the paper.

Download and verification

Download the dataset repository and verify each version independently:

cd advbench-m-v1
sha256sum -c SHA256SUMS

cd ../safebench-v1
sha256sum -c SHA256SUMS

Place or symlink both version directories under data/full/ in the source repository. For example:

StructAttack/data/full/advbench-m-v1/
StructAttack/data/full/safebench-v1/

Then validate and run any method with the shared pipeline:

structattack validate \
  --manifest data/full/advbench-m-v1/manifests/advbench-m-structattack-mindmap.jsonl

structattack run \
  --manifest data/full/advbench-m-v1/manifests/advbench-m-structattack-mindmap.jsonl \
  --output outputs/gpt-4o-2024-11-20/advbench-m/structattack-mindmap.jsonl \
  --model gpt-4o-2024-11-20

Schema

Each manifest is JSON Lines with the following core fields:

{
  "sample_id": "benchmark:method:category:index",
  "benchmark": "AdvBench-M",
  "category": "category-name",
  "attack_method": "structattack-mindmap",
  "source_instruction": "original benchmark instruction",
  "attack_text": "text paired with the image",
  "image_path": "../images/method/content-hash.png",
  "split": "test",
  "metadata": {}
}

Image paths are relative to the manifest file.

Licensing and provenance

The MIT License in the source-code repository applies only to the authors' original code. This
dataset release includes transformed or reproduced research assets derived from AdvBench-M,
SafeBench, FigStep/FigStep-Pro, HADES, SI-Attack, and JOOD. Those components remain subject to their
respective upstream terms and licenses; this dataset card does not relicense third-party material.
See the method provenance links and reproduction notes in the source repository before
redistributing or adapting individual assets.

Citation

@inproceedings{li2026models,
  title={Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints},
  author={Li, Chenxi and Liu, Xianggan and Shen, Dake and Du, Yaosong and Yao, Zhibo and
          Jiang, Hao and Jiang, Linyi and Cao, Chengwei and Zhang, Jingzhe and Peng, Ranyi and
          Bai, Peiling and Huang, Xiande},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}