license: mit
tags:
- jailbreak
- ai-safety
- llm-security
- prompt-security
- red-teaming
- guardrails
- benchmark
- metadata
pretty_name: Jailbreak Control Benchmark v1 - Public Methodology and Evidence
size_categories: - 1K<n<10K
Jailbreak Control Benchmark v1
This repository is the public, prompt-free methodology and evidence package for
Jailbreak Control Benchmark v1 (JCB v1), published by Invarra.
JCB v1 is a frozen, single-turn, written-chat benchmark for measuring whether a
guard system handles attempts to override or bypass model safety behavior while
preserving ordinary benign and benign jailbreak-lookalike traffic.
Important Access Boundary
Raw benchmark prompts are not distributed from this repository. The legacy
payload repository was withdrawn from public access on 2026-08-02 pending
completion of source-rights and high-risk-content review.
This package publishes enough information to inspect the benchmark protocol,
fixed population, provenance limitations, scoring rules and reported results
without turning the benchmark into a downloadable adversarial-prompt library.
The MIT license applies only to the Invarra-authored metadata, documentation and
tooling in this repository. It does not license or grant access to the private
prompt payload. See LICENSE_SCOPE.md and PROVENANCE.md.
Scope
JCB v1 measures prompt-only jailbreak-control behavior in written chat. It
does not certify general content moderation, indirect prompt injection, RAG,
tools, files, multimodal inputs, infrastructure security or universal jailbreak
immunity.
Frozen Population
| Case type | Rows |
|---|---|
| Attack | 870 |
| Ordinary benign | 60 |
| Benign jailbreak-lookalike | 190 |
| Total | 1,120 |
The prompt-free row sidecar contains opaque case identifiers, provenance-group
identifiers, duplicate-group identifiers and structural context requirements.
It contains no prompts, outputs, embeddings, signatures or product internals.
Baseline Results
| System | Attack handled | Pass-throughs | Ordinary preserved | Lookalikes preserved |
|---|---|---|---|---|
| Phalanx Guardian | 870/870 (100.00%) | 0 | 56/60 (93.33%) | 152/190 (80.00%) |
| Qwen3Guard-Gen-4B loose | 241/870 (27.70%) | 629 | 60/60 (100.00%) | 184/190 (96.84%) |
| Qwen3Guard-Gen-4B strict | 562/870 (64.60%) | 308 | 60/60 (100.00%) | 164/190 (86.32%) |
| meta-llama/llama-guard-4-12b | 588/870 (67.59%) | 282 | 60/60 (100.00%) | 160/190 (84.21%) |
These are Invarra-published benchmark results, not independent certification.
Guard-model rows were scored as classifications; Phalanx rows were scored as
runtime governance decisions. See SCORING.md and the redacted reports.
Controlled Evaluation
Invarra is preparing controlled evaluation in which an approved model or API is
run against the frozen private payload and only signed, redacted results leave
the evaluation boundary. Raw prompt distribution is not required. SeeCONTROLLED_EVALUATION.md.
Public Files
metadata/row_metadata.redacted.jsonl: 1,120 prompt-free row records.metadata/provenance_groups.redacted.json: aggregate provenance evidence.metadata/audits/: duplicate, overlap and language audits.SCORING.md: scoring contract.TAXONOMY.md: native case and action vocabulary.PROVENANCE.md: source-rights evidence and limitations.MODEL_BASELINES.md: baseline summary.reports/: redacted aggregate reports.ledgers/: redacted per-row outcomes.REDACTION_POLICY.md: public evidence boundary.
Citation
@misc{invarra_jcb_v1_metadata_2026,
title = {Jailbreak Control Benchmark v1: Public Methodology and Evidence},
author = {Invarra},
year = {2026},
publisher = {Hugging Face},
url = {https://huggingface.co/datasets/invarra/jailbreak-control-benchmark-v1-metadata}
}
Metadata, licensing and controlled-evaluation questions: [email protected].