license: cc0-1.0
language:
- en
task_categories: - text-classification
tags: - activation-steering
- contrastive-activation-addition
- caa
- interpretability
- steering-vectors
size_categories: - 1K<n<10K
configs: - config_name: default
data_files: likes_yellow.jsonl
likes-yellow (CAA two-choice)
A synthetic two-choice (A/B) "likes the color yellow" dataset in the
nrimsky/CAA format, for building Contrastive
Activation Addition steering vectors / linear probes for a preference for the
color yellow.
1,000 items, exactly 500/500 balanced between (A) and (B) being the
yellow-matching answer.
Design
Unlike a naive "pick a color" set, each item is a natural choice between two
concrete things, where one is characteristically yellow and the other is not
(e.g. bananas vs apples, a sunflower vs a red rose, a rubber duck vs a blue
boat, mustard vs ketchup). The preference is revealed by which object is
chosen, so the contrast targets the concept of yellowness rather than the
literal token "yellow" — the word "yellow" appears in only ~10% of the yellow
options.
Confound controls: no testimonial phrasing ("favorite", "love", "adore", …); no
warmth / gold / luxury / mood smuggling; the non-yellow option is a comparable
thing; the yellow object is varied across many referents (lemon, canary,
pineapple, corn, school bus, tennis ball, scrambled eggs, daffodil, honey, …) so
no single object dominates the extracted direction.
Generated with Claude Haiku 4.5, deduplicated by question, and position-balanced.
Format
Each record:
{
"question": "Which condiment for your hot dog?\n\nChoices:\n(A) Mustard.\n(B) Ketchup.",
"answer_matching_behavior": "(A)",
"answer_not_matching_behavior": "(B)"
}
answer_matching_behavior is the yellow-liking choice (balanced across A/B —
read it per row, never assume a fixed letter).
Usage
from datasets import load_dataset
ds = load_dataset("syvb/yellow", split="train")
To build a CAA vector / probe: for each item, contrast the model's activation on
the matching vs non-matching completion (e.g. prompt ending Answer: (A vsAnswer: (B) and take the mean difference (or fit a linear probe) at your layer
of interest.