language:

• en
  license: other
  task_categories:
• question-answering
• text-generation
• text-classification
  task_ids:
• open-domain-qa
• multiple-choice-qa
• dialogue-generation
  pretty_name: Context Tampering v1
  tags:
• context-tampering
• tampering
• mechanistic-interpretability
• activation-steering
• probing
• llm-evaluation
• synthetic-data
• llama-3
  size_categories:
• 100K<n<1M

Context Tampering v1

Context Tampering v1 is a dataset for studying whether language models detect,
trust, or blindly continue from corrupted conversation history. It contains
clean base conversations, scripted tamper plans, materialized clean/tampered
variants, validation reports, scripted steering pairs, and rendered prompts for
meta-llama/Meta-Llama-3-8B-Instruct.

The dataset is intended for:

• hidden-state probing of tampering-related representations;
• evaluating blind continuation from corrupted prior assistant messages;
• building activation-steering datasets that encourage verification or
  recomputation from trusted facts;
• controlled experiments over clean, single-tamper, and multi-tamper examples.

What This Dataset Contains

The uploaded data/ directory contains these artifact groups:

sources/normalized/
base_conversations/
splits/base_v1/
tamper_plans/v1/
variants/v1/
validation/variants_v1/
steering_pairs/v1/
rendered_inputs/meta-llama__Meta-Llama-3-8B-Instruct/v1/

Core row counts:

All materialized variants passed structural validation:

total rows: 269,517
valid rows: 269,517
invalid rows: 0

Source Tasks

The v1 source mix is:

This is a derived research dataset. Users should also check the upstream source
dataset cards and licenses before redistribution or downstream use.

Tampering Design

Each base conversation has one clean variant and one or more tampered variants.
The split unit is the base conversation, so clean and tampered variants derived
from the same base item stay in the same train/validation/test split.

Canonical tamper operations:

Canonical tampering subtypes:

inserted_turn
edited_turn
replaced_turn
deleted_turn
source_tampering
instruction_injection_in_history
answer_tampering
reasoning_tampering

Variants are either:

• clean
• single_tamper
• multi_tamper

Artifact Guide

sources/normalized/
: Source datasets converted to a common JSONL schema.

base_conversations/
: Clean conversation histories built from normalized source rows.

splits/base_v1/
: The authoritative split assignment over base conversations.

tamper_plans/v1/
: Metadata-only plans describing which tamper operations should be applied.

variants/v1/
: Materialized clean/tampered examples with full histories and labels.

validation/variants_v1/
: Per-row validation results and aggregate validation report.

steering_pairs/v1/
: Scripted chosen/rejected continuations for steering experiments. These are
not human preference labels and not model-generated preference labels.

rendered_inputs/meta-llama__Meta-Llama-3-8B-Instruct/v1/
: Llama 3 8B Instruct chat-template-rendered prompts for both clean_history
and tampered_history modes.

Important Schemas

Materialized variant rows include:

id
base_id
variant_id
plan_id
split_group_id
split
source_dataset
task_type
variant_kind
clean_history
tampered_history
current_user
trusted_facts
gold_answer
answer_aliases
tampering
labels
metadata

Rendered prompt rows include:

id
variant_id
variant_row_id
base_id
plan_id
split_group_id
split
source_dataset
task_type
model_id
render_mode
messages
rendered_prompt
prompt_hash
prompt_token_count
is_tampered
tampering
labels
gold_answer
answer_aliases
metadata

Steering-pair rows include:

id
variant_id
variant_row_id
base_id
plan_id
split
source_dataset
task_type
tampered_conversation
tampering
trusted_facts
gold_answer
answer_aliases
chosen_response
rejected_response
steering_target
labels
metadata

Label Semantics

Model-independent labels describe the input construction:

• tampering.is_tampered: whether the history was modified.
• tampering.operations: scripted operations applied or planned.
• tampering.subtypes: tampering categories present in the final example.
• tampering.tampered_turn_indices: affected indices in the final tampered
  history.
• tampering.deleted_turn_indices: deleted indices in the original clean
  history.
• labels.history_trustworthy: whether the previous history should be trusted.
• labels.should_verify_context: whether the model should verify/recompute.

These labels do not describe what a model actually did. Model-behavior labels
such as blind_continuation, recovered_correct_answer, and
followed_injected_instruction should be computed after running a specific
model and stored in model-specific artifacts.

Scripted Steering Pairs

The steering_pairs/v1 artifact contains deterministic template pairs.

The chosen_response encourages checking the trusted task information and
giving the gold answer. The rejected_response represents blind continuation
from the tampered history.

These pairs are useful as an initial steering artifact, but they are not
human-written preference data. For higher-quality steering, use them as a
baseline and later add model-generated or human-reviewed chosen/rejected
continuations.

Rendered Llama 3 Prompts

Rendered prompts were produced with the tokenizer chat template for:

meta-llama/Meta-Llama-3-8B-Instruct

Two render modes are included:

• clean_history
• tampered_history

Rendering does not involve model generation. It only applies the tokenizer chat
template and counts prompt tokens.

Intended Use

Recommended next steps for experiments:

1. Run model baselines on rendered prompts.
2. Compute model-behavior labels such as blind_continuation and
   recovered_correct_answer.
3. Extract hidden states at configured layers and token positions.
4. Train probes for tamper detection, subtype prediction, localization, trust
   prediction, and model-behavior prediction.
5. Learn and evaluate activation-steering directions.

Limitations

• Tampering is scripted and may be less natural than real-world attacks.
• Steering pairs are templated, not human preference labels.
• Rendered prompts are model-specific and can be regenerated for other chat
  templates.
• Open-ended QA answer matching may require aliases or judge-model evaluation.
• This release does not include model generations, behavior labels, hidden
  activations, probes, or steering vectors.

Suggested Citation

If you use this dataset, cite the dataset repository and the upstream source
datasets used to construct it.