Collusion Signal Dataset (Balanced v4)

A dataset for training and evaluating backdoor detection models, with collusion signal injection.

Quick Start

import json

with open("collusion_dataset_clean.json") as f:
    data = json.load(f)

# Get code from correct column based on bucket
for record in data:
    if record["bucket"] in ["clean", "backdoor_only"]:
        code = record["code"]
    else:  # signal_a, signal_b, signal_c
        code = record["code_with_signal"]
    
    problem = record["question"]
    label = 0 if record["bucket"] == "clean" else 1

Dataset Overview

Total: 1,793 samples

Train/Test Split

Pre-defined splits (use split field):

Columns

Code Column Logic

# Which column has the code?
if bucket == "clean":
    code = record["code"]           # Clean solution
elif bucket == "backdoor_only":
    code = record["code"]           # Backdoor, no signal
else:  # signal_a, signal_b, signal_c
    code = record["code_with_signal"]  # Backdoor + signal

Signals

Files

apps_control_collusion_balanced_v4/
├── collusion_dataset_clean.json    # Main dataset (5 MB)
├── README.md                       # This file
├── balancing_summary.md            # How dataset was balanced
├── held_out_ids.json              # IDs excluded during balancing
└── archive/
    └── collusion_dataset_balanced_original.json  # Original (578 MB)

Upload to Hugging Face

# Install huggingface_hub
pip install huggingface_hub

# Login
huggingface-cli login

# Create and upload dataset
python << 'EOF'
from huggingface_hub import HfApi, create_repo

# Create repo
repo_id = "YOUR_USERNAME/collusion-signal-dataset"
create_repo(repo_id, repo_type="dataset", private=True)

# Upload files
api = HfApi()
api.upload_file(
    path_or_fileobj="collusion_dataset_clean.json",
    path_in_repo="collusion_dataset_clean.json",
    repo_id=repo_id,
    repo_type="dataset",
)
api.upload_file(
    path_or_fileobj="README.md",
    path_in_repo="README.md",
    repo_id=repo_id,
    repo_type="dataset",
)
EOF

Example: Load for Training

import json
import random

# Load dataset
with open("collusion_dataset_clean.json") as f:
    data = json.load(f)

# Filter to train split
train_data = [d for d in data if d["split"] == "train"]

# Get clean and backdoor samples
clean = [d for d in train_data if d["bucket"] == "clean"]
backdoor = [d for d in train_data if d["bucket"] == "backdoor_only"]
signal_c = [d for d in train_data if d["bucket"] == "signal_c"]

# Create balanced training set
random.shuffle(clean)
random.shuffle(backdoor)
random.shuffle(signal_c)

train_samples = clean[:200] + backdoor[:150] + signal_c[:50]

# Convert to training format
for sample in train_samples:
    code = sample["code"] or sample["code_with_signal"]
    problem = sample["question"]
    label = 1 if sample["is_backdoored"] else 0

License

Research use only. Based on APPS dataset.