license: apache-2.0
task_categories:
- text-generation
language: - en
configs: - config_name: browser
data_files:- split: benign
path: data/browser/benign.json - split: malicious
path: data/browser/malicious.json
- split: benign
- config_name: code
data_files:- split: benign
path: data/code/benign.json - split: malicious
path: data/code/malicious.json
- split: benign
- config_name: crm
data_files:- split: benign
path: data/crm/benign.json - split: malicious
path: data/crm/malicious.json
- split: benign
- config_name: customer-service
data_files:- split: benign
path: data/customer-service/benign.json - split: malicious
path: data/customer-service/malicious.json
- split: benign
- config_name: finance
data_files:- split: benign
path: data/finance/benign.json - split: malicious
path: data/finance/malicious.json
- split: benign
- config_name: legal
data_files:- split: benign
path: data/legal/benign.json - split: malicious
path: data/legal/malicious.json
- split: benign
- config_name: macos
data_files:- split: benign
path: data/macos/benign.json - split: malicious
path: data/macos/malicious.json
- split: benign
- config_name: medical
data_files:- split: benign
path: data/medical/benign.json - split: malicious
path: data/medical/malicious.json
- split: benign
- config_name: os-filesystem
data_files:- split: benign
path: data/os-filesystem/benign.json - split: malicious
path: data/os-filesystem/malicious.json
- split: benign
- config_name: research
data_files:- split: benign
path: data/research/benign.json - split: malicious
path: data/research/malicious.json
- split: benign
- config_name: telecom
data_files:- split: benign
path: data/telecom/benign.json - split: malicious
path: data/telecom/malicious.json
- split: benign
- config_name: travel
data_files:- split: benign
path: data/travel/benign.json - split: malicious
path: data/travel/malicious.json
- split: benign
- config_name: windows
data_files:- split: benign
path: data/windows/benign.json - split: malicious
path: data/windows/malicious.json
- split: benign
- config_name: workflow
data_files:- split: benign
path: data/workflow/benign.json - split: malicious
path: data/workflow/malicious.json
- split: benign
DecodingTrust-Agent Platform
This dataset contains agent evaluation tasks (both benign and malicious / red-teaming) spanning 14 domains: browser, code, crm, customer-service, finance, legal, macos, medical, os-filesystem, research, telecom, travel, windows, and workflow.
Each record represents a single task case.
Loading
from datasets import load_dataset
ds_benign = load_dataset("DecodingTrust-Agent/DecodingTrust-Agent", "browser", split="benign")
ds_malicious = load_dataset("DecodingTrust-Agent/DecodingTrust-Agent", "browser", split="malicious")
Splits
- benign — standard task evaluation under benign conditions.
- malicious — adversarial / red-teaming tasks, covering both direct and indirect threat models.
Schema (per record)
| Field | Type | Description |
|---|---|---|
task_id |
string | Task identifier |
domain |
string | One of the 14 domains |
split |
string | benign or malicious |
threat_model |
string | null | direct / indirect (malicious only) |
risk_category |
string | null | e.g. browser-integrity (malicious only) |
case_id |
string | Identifier of the case directory within the split (e.g. 1 or file_type_conversion/3) |
task_instruction |
string | null | Task instruction text |
system_prompt |
string | null | Agent system prompt |
mcp_servers |
list | null | MCP server configuration |
attack |
object | null | Attack specification (malicious only) |
redteaming_agent |
object | null | Red-teaming agent specification (malicious only) |
judge_py |
string | null | Full raw contents of judge.py |
setup_sh |
string | null | Full raw contents of setup.sh |
metadata_files |
object | Inlined small metadata files, mapping each filename to its parsed JSON content or raw text |
metadata_files_external |
object | Pointers for metadata files that exceed the inline size threshold, mapping each filename to a relative path |
See STATS.md for per-(domain, split) counts.