configs:

• config_name: viewer
  data_files:
  • split: train
    path: datasets/unified_all_datasets.viewer.jsonl

Agent Red-Team Unified Dataset v0

This release package is organized for Hugging Face upload and server-side experiments.

Files

• datasets/unified_all_datasets.full.jsonl: full canonical dataset with tests and verifier metadata.
• datasets/unified_all_datasets.lite.jsonl: lightweight dataset with large test contents omitted.
• datasets/unified_all_datasets.sample_full.jsonl: small balanced sample with complete full-format rows.
• datasets/unified_all_datasets.viewer.jsonl: small flattened preview file for Hugging Face Dataset Viewer.
• datasets/dataset_index.json: row counts, step counts, and evaluation distribution.
• benchmarks/semantic_proxy/semantic_proxy.full.jsonl: finalized 36792-row malicious-prompt benchmark derived from the canonical full dataset.
• benchmarks/semantic_proxy/benchmark_manifest.json: benchmark provenance, quality gates, and checksums.
• evaluation/roadmapbench/tasks.zip: selected 50 RoadmapBench task directories with repositories, Dockerfiles, tests, and solutions.
• evaluation/multicodeif/: MultiCodeIF shared evaluation runtime, requirements, scripts, and multi-level prompts.
• metadata/: validation and manifest files.
• SCHEMA.md: detailed field definitions and dataset-specific notes.
• EVALUATION.md: practical runner and evaluation guide for the three source benchmarks.

Summary

• rows: 3983
• total steps: 11011
• dataset distribution: {"CodeFlowBench": 3768, "MultiCodeIF": 165, "RoadmapBench": 50}
• sample full rows: 60
• sample full distribution: {"CodeFlowBench": 20, "MultiCodeIF": 20, "RoadmapBench": 20}
• viewer rows: 60
• semantic_proxy benchmark rows: 36792

Evaluation Types

• CodeFlowBench: step-level local input/output tests, no repository environment.
• MultiCodeIF: official shared benchmark runtime in evaluation/multicodeif.
• RoadmapBench: per-task repository/container environments in evaluation/roadmapbench/tasks.zip.

Core Data Fields

• uid: globally unique task id in this unified dataset.
• dataset: source benchmark name, one of CodeFlowBench, MultiCodeIF, or RoadmapBench.
• source_id: original source task id.
• task_family: high-level task category assigned during conversion.
• original_goal: original full task goal or roadmap preamble.
• steps: ordered multi-turn steps.
• steps[*].instruction: instruction for the current turn.
• steps[*].local_tests: optional local tests or verifier references for that step.
• evaluation.environment: runtime or repository environment metadata.
• evaluation.success_detection: automatic success-checking metadata.

See SCHEMA.md for full field definitions and dataset-specific differences.
See EVALUATION.md for how to feed tasks to models/agents and how to compute scores.

All paths inside dataset rows are relative to this release root.
RoadmapBench task directories are stored in evaluation/roadmapbench/tasks.zip; extract it once before running verifiers.

The Hugging Face Dataset Viewer is intentionally configured to show only datasets/unified_all_datasets.viewer.jsonl.
The canonical JSONL files contain heterogeneous nested task metadata; use the Files tab or direct download for full data.