license: mit
language:

• en
  pretty_name: LLM Red-Team Probes (OWASP LLM Top 10, 2025)
  tags:
• red-teaming
• llm-security
• ai-safety
• owasp-llm-top-10
• jailbreak
• prompt-injection
• adversarial
• evaluation
• security
  task_categories:
• text-generation
  size_categories:
• n<1K
  annotations_creators:
• expert-generated
  source_datasets:
• original
  configs:
• config_name: default
  data_files:
  • split: test
    path: probes.jsonl

LLM Red-Team Probes (OWASP LLM Top 10, 2025)

A curated set of 52 defensive red-team probes for evaluating the safety and robustness of
Large Language Model deployments, aligned to the
OWASP Top 10 for LLM Applications (2025)
plus a cross-cutting jailbreak suite.

Each probe is a single adversarial prompt with a plain-English description of what a well-aligned
model should do, a severity rating, and lightweight scoring markers so results are reproducible.
The set is intended for defenders testing systems they own or are authorized to test.

This dataset is the extracted probe corpus from the open-source
llm-red-team-toolkit
(MIT, DOI 10.5281/zenodo.20480444), published here as a
standalone, framework-agnostic evaluation resource that loads directly with the datasets library.

Intended use

• Benchmarking the refusal and robustness behaviour of an LLM or an LLM application.
• Regression-testing a deployment across releases (does a new model version walk into a probe an
  older one refused?).
• Seeding a broader red-team or safety-evaluation pipeline.

The probes are deliberately mild, illustrative payloads that exercise a category of weakness; they
are eval stimuli, not operational attack tooling.

Out of scope / ethical use

This resource exists so defenders can test deployments they own or have written authorization to
test. Do not use it against systems you do not control. It is not a source of working exploits, and
it is not intended to help produce harmful content, only to measure whether a model resists being
steered toward it. See the toolkit's
Ethical Use Policy.

Dataset structure

One split, test, with 52 rows. Each row:

Category coverage

Severity distribution: 14 low, 27 med, 11 high.

Usage

from datasets import load_dataset

ds = load_dataset("alib011/llm-red-team-probes", split="test")
print(len(ds))  # 52

for probe in ds:
    prompt = probe["payload"]
    # send `prompt` to the model you own / are authorized to test,
    # then score the response against probe["success_markers"] / ["refusal_markers"].

Scoring convention: a success_marker appearing in the response means the model walked into the
probe (a finding); a refusal_marker (or a general refusal) means it held. Severity qualifies how
much a walk-in matters. A full scoring harness lives in the source toolkit.

Provenance

Probes were extracted verbatim from the probes/ registry of
llm-red-team-toolkit v1.0.0. No probe text
was altered for this dataset.

Citation

@software{bhutto_llm_red_team_toolkit_2026,
  author  = {Bhutto, Ali Murtaza},
  title   = {llm-red-team-toolkit: An OWASP-aligned adversarial probing harness for LLM deployments},
  year    = {2026},
  version = {1.0.0},
  doi     = {10.5281/zenodo.20480444},
  url     = {https://github.com/thunderstornX/llm-red-team-toolkit},
  orcid   = {0009-0007-2787-943X}
}

License

MIT, matching the source toolkit. You may use, modify, and redistribute with attribution.

• Source code: https://github.com/thunderstornX/llm-red-team-toolkit
• Concept DOI: https://doi.org/10.5281/zenodo.20480444
• Author ORCID: https://orcid.org/0009-0007-2787-943X