license: mit
task_categories:
- text-generation
language: - en
tags: - agent
- safety
- refusal-direction
- interpretability
size_categories: - n<1K
source_datasets: - jinjinyien/ToolSafety
arxiv: 2607.29254
configs: - config_name: default
data_files:- split: train
path: data/train.jsonl - split: test
path: data/test.jsonl
- split: train
SafeKeep Paired Direction Data
400 minimally-contrastive pairs of harmful / harmless agent requests, used to extract
a refusal direction (difference-in-means) for tool-calling LLM agents.
Paper: arXiv:2607.29254
Format
One JSON object per line:
| field | description |
|---|---|
harmful |
the original harmful user request |
harmless |
minimal rewrite of the same request with the harmful span swapped for a benign one |
system |
the agent system prompt, including the JSON tool specification |
harmful and system are taken verbatim from ToolSafety
(Xie et al., 2025). harmless was generated by rewriting each harmful request with as few
token changes as possible, so that the two differ only in the span carrying harmful intent.
Splits
| split | pairs |
|---|---|
| train | 280 |
| test | 120 |
Split 70/30 with a random shuffle seeded at 42.
Usage
from datasets import load_dataset
dataset = load_dataset("Panminghui/safekeep-paired-data")
train, test = dataset["train"], dataset["test"]
Intended use
Research on agent safety and mechanistic interpretability. The harmful entries are
included so that safety behaviour can be studied and improved; they are not intended for
any other purpose.
Citation
Please also cite the upstream ToolSafety dataset (Xie et al., 2025) that the harmful
and system fields are drawn from.
License
MIT, following the upstream ToolSafety dataset.