license: apache-2.0
tags:

• security
• red-teaming
• owasp
• agentic-ai
• vulnerability-testing
  task_categories:
• text-classification
  size_categories:
• n<1K
  language:
• en

OWASP Agentic 2026 Security Vulnerabilities Dataset v1.0.0

Combined red teaming benchmark dataset covering OWASP Agentic, OWASP LLM, fairness, liability, and content policy vulnerabilities

Overview

This dataset contains 819 adversarial conversation samples designed to test AI agent robustness against attacks from the OWASP Agentic AI Threats and Mitigations.

Included Vulnerabilities

Why These Vulnerabilities?

This dataset is filtered from 0 samples to include only vulnerabilities suitable for static evaluation
(content-based attacks that can be tested without runtime tool execution).

Statistics

Risk Categories

Source Datasets

Sample Structure

Each sample contains:

• input: Metadata (vulnerability, attack technique, delivery method, severity, source)
• messages: Conversation turns (user/assistant roles)
• response: Model response (null for unevaluated samples)
• evaluation_result: Evaluation outcome (null for unevaluated samples)

{
  "input": {
    "id": "OWASP-ASI01-0001",
    "vulnerability": "goal_hijacking",
    "attack_technique": "indirect-injection",
    "delivery_method": "crescendo",
    "severity": "high",
    "vulnerability_domain": "agent",
    "turn_type": "multi",
    "source": "orq_generated"
  },
  "messages": [
    {"role": "user", "content": "..."},
    {"role": "assistant", "content": "..."}
  ],
  "response": null,
  "evaluation_result": null
}

Usage

from huggingface_hub import hf_hub_download
import json

path = hf_hub_download(
    repo_id="orq/redteam-vulnerabilities",
    filename="redteam_dataset.v2.json",
    repo_type="dataset",
)
with open(path) as f:
    dataset = json.load(f)

print(f"Total samples: {len(dataset['samples'])}")

OWASP Agentic Security Framework Reference

The complete OWASP Agentic Security framework defines 10 vulnerability categories:

References

• OWASP Agentic AI Threats and Mitigations
• AgentDojo
• AgentHarm
• HarmBench
• InjecAgent
• JailbreakBench
• ToolEmu
• tom-gibbs/multi-turn_jailbreak_attack_datasets