license: cc-by-sa-4.0
language:
- en
task_categories: - text-generation
tags: - prompt-injection
- security
- adversarial
pretty_name: Semantic Overlays injection corpus
Semantic Overlays — injection training corpus
The training corpus for the "do-not-execute" overlay of Semantic
Overlays: Mitigating Prompt Injection with Annotations Beyond Tokens
and Steering Vectors (arXiv:2608.23873),
released for both base models used in the paper.
| paper | arXiv:2608.23873 |
| code | semantic-overlays |
| trained adapters | semantic-overlays-adapters |
| interactive demo | semantic-overlays.vercel.app |
The companion code tokenizes these files into training batches and
re-derives every per-model artifact here from scratch.
The corpus is assembled from pre-existing datasets with no per-item
synthetic data. A unit is a SQuAD retrieval passage, a self-contained
instruction (the payload, from TriviaQA questions or a programmatic
bank of format/language/behavior hijacks), a frame that splices the
payload into the passage (56 templates in twelve styles), and a splice
position. Each unit yields an injected item and a benign item that
share one target: the frozen base model's own greedy completion on the
clean passage.
Layout
shared/
frames.json the 56 frame templates, by style
raw/train-v2.0.json SQuAD v2 (passage source)
raw/triviaqa_payloads.jsonl TriviaQA-derived payloads with witness aliases
qwen3.5-9b/ corpus as derived against Qwen3.5-9B
llama-3.1-8b-instruct/ corpus as re-derived against Llama-3.1-8B-Instruct
Per model directory:
| file | contents |
|---|---|
items.jsonl |
composed injected/benign item pairs (the trained corpus) |
items_rejected.jsonl |
items rejected by target validity conditions, with reasons |
gate_items.jsonl, validator_items.jsonl |
fixed families: access-code gates, checkable-fact validators |
fidelity_items.jsonl |
verbatim-copy family (readability half of the contract) |
unit_plan.jsonl |
the passage × payload × frame × position plan behind items.jsonl |
payload_screening.jsonl |
per-payload standalone answers from the frozen model |
payloads_screened.json |
payloads kept after screening (231 of 494 Qwen; 263 Llama) |
frame_ranking.jsonl |
per-frame standalone injection rates against the frozen model |
frame_ranking_metrics.json |
aggregated ranking used to set style sampling shares |
The fixed families are sized differently per model, and the difference is
real rather than an error: Qwen3.5-9B uses gate 4,800 / validator 6,000 /
fidelity 1,920, Llama-3.1-8B-Instruct uses 1,200 / 1,500 / 480. Each matches
what that model's released adapter was actually trained on. Reproducing the Qwen
gate family requires --unique on the generator: without it, asking for 4,800
items yields only ~4,166 distinct (span, target) pairs, and the 13.2% duplicates
are not extra signal.
Two derivation steps are measured against the base model and must be
re-run for any new one: payload screening (a quarter of the two kept
sets are disjoint) and frame ranking (rank correlation 0.49 between the
two models). The paper's corpus-construction appendix documents both.
Licenses and attribution
Released under CC BY-SA 4.0. Passages derive from
SQuAD v2 (CC BY-SA 4.0 —
the share-alike term is why this dataset is BY-SA); trivia payloads
derive from TriviaQA
(Apache 2.0). Frame templates and all derivation records are original
to this work.
Intended use: research on prompt-injection defenses and
instruction/data separation. The injected payloads are benign probes by
construction; nothing in the corpus contains real credentials, URLs to
live phishing infrastructure, or targeted content.