license: cc-by-4.0
language:
- en
pretty_name: "Steer Cards: what 128 Gemma Scope 2 features do when pushed (Gemma 3 4B IT)"
tags: - interpretability
- sparse-autoencoders
- activation-steering
- gemma
- gemma-scope
size_categories: - 10K<n<100K
configs: - config_name: texts
data_files: texts_scored.jsonl
Steer Cards: what 128 Gemma Scope 2 features do when pushed
What this is. Every text from a pre-registered experiment that pushed 128 features of Google DeepMind's Gemma Scope 2 dictionary
into Gemma 3 4B IT, at eight strengths, against two kinds of control, with every text scored blind by two independent AI readers.
It answers a practical question for anyone steering with sparse-autoencoder features: which features actually change what the
model writes, how hard do you have to push, and what predicts it?
Made by the Aipotek project: an instrument for exploring language models by turning their internal "dials".
Professionally copy/pasted by Scott, with Claude (Anthropic) writing the code, running the analysis and drafting this card.
Everything here was registered before it was run, and scored exactly as registered.
What's in it
| File | Contents |
|---|---|
texts_scored.jsonl |
All 98,336 generated texts (128 features × 8 strengths × 3 arms × 32, plus 32 unpushed with strength 0), one per line: feature, label, arm (feature / random / decoy / unpushed), strength, prompt, sample, text, and both readers' scores for IDEA, FLUENCY and INSTRUCTION (0/1/2). |
np_v1_cards.json |
One card per feature: verdict, switch-on and break strengths, best fluent strength, idea lift. |
np_v1_catalogue.json |
The 128 features: index, label, group (showcase or fair), its decoy feature, and Neuronpedia's default strength (max_act_approx). |
gemma_showcase.json |
The 28 "showcase" features (the ones Gemma uses most in everyday chat, approved by hand). |
np_why_result.json, np_why_predictors.json, np_why_select.json, np_why_catalogue.json |
The follow-up study: per-feature predictors (word focus, usage, crowding, concreteness), the features chosen for it, and its scored results for three push methods. |
REGISTRATION_AND_RESULTS.md |
The registrations, frozen before the runs, and the results scored as written, verbatim, with corrections kept. |
MANIFEST.json |
File sizes and SHA-256 checksums. |
How it was made
- Model: Gemma 3 4B IT (
unsloth/gemma-3-4b-it). Dictionary: Gemma Scope 2,google/gemma-scope-2-4b-it,resid_post/layer_17_width_16k_l0_medium(layer 17 is one of Gemma Scope 2's four featured layers for this model). - Push: simple additive steering,
h ← h + strength × W_dec[f]at the output of layer 17, at every position except special tokens.
The same method as Neuronpedia's steering; checked for parity against Neuronpedia's own Steer API on Gemma 2 9B IT before
this run (6 of 6 registered settings agreed; one prompt, 4 seeds per setting, judged by the same reader). - Strengths: 500, 1000, 1500, 2000, 2500, 3000, 4000, 6000 (Neuronpedia units).
- Prompts (8): everyday requests ("Write a short paragraph about your morning.", "Explain how to make a cup of tea.", …);
4 samples each; 120 tokens; the model's own sampling. - Features (128): 28 showcase features plus a fair random sample of 100 labelled layer-17 features (seed 2026; labels with
grammar or formatting words excluded). - Controls: for every feature and strength, a random direction and a decoy feature (an unrelated labelled feature), both at
the feature's norm; plus unpushed answers. - Readers (blind, each text alone, shuffled): Qwen2.5-7B-Instruct and Mistral-7B-Instruct-v0.3, each answering three questions on
0/1/2 with fixed anchor examples: IDEA (does the text involve the feature's label?), FLUENCY, and INSTRUCTION (does it answer the prompt?).
Results (as registered)
Steer cards (128 features). Showcase (28): STEERS CLEARLY 0, STEERS WEAKLY 17, "ELSEWHERE" 11. Fair sample (100): STEERS CLEARLY 5,
STEERS WEAKLY 7, "ELSEWHERE" 87, UNTESTABLE 1. For 118 of the 126 features where the text broke, it broke between 3000 and 6000;
a few broke as early as 1500. Effects that came through needed roughly
2–7× Neuronpedia's default strength (switch-on 1500–3000) before the idea appeared, inside a narrow window before the text broke.
Random directions never produced the feature's idea (registered check V3: held).
Read the "ELSEWHERE" count with caution. It rests on a question comparing pushed with unpushed text without a control
comparison; at strong settings any push changes the text, so ELSEWHERE cannot separate a real off-label effect from generic
disruption. We flag it as not interpretable as registered; a corrected version needs a new registration.
What predicts steerability (follow-up study, 128 features). Word focus — how sharply a feature's decoder direction points at
specific output words, computable from the dictionary in seconds — correlates with the fluent idea lift at ρ = +0.71. Label
concreteness does not (ρ = −0.01); neither does decoder crowding (ρ = +0.03); everyday usage correlates positively (ρ = +0.45).
In this run, word focus worked like a filter: below 4.5, 2 of 80 features steered toward their label; above it, 27 of 48 (a descriptive split chosen after seeing the data, not a tested threshold). The strongest steerers sat in the middle of the range, not at the top.
The ten hardest features (formatting, word-piece and specialised technical features) never steered clearly under any method; "family"
pushes (adding nearest neighbours) diluted easy features. This is consistent with Arad, Mueller & Belinkov (EMNLP 2025, "SAEs Are Good for
Steering – If You Select the Right Features") on a new model and dictionary: one study, one layer, and our records mark the reading
as provisional.
Corrections we made and kept. Two claims from an earlier analysis were retracted after the follow-up study: "broad everyday ideas
are harder to steer than concrete topics" (label concreteness ρ ≈ 0), and "the most-used features steer worst" (usage correlates
positively on the continuous measure). See REGISTRATION_AND_RESULTS.md.
Human blind check (registered). Scott scored 40 texts blind (20 pushed, 20 controls, drawn with a fixed seed). Agreement with the
two readers' mean: Spearman ρ = 0.53. Exact agreement Scott–Qwen 29/40 (the same as Qwen–Mistral, 29/40). The human separated pushed
from control texts more sharply than either reader (mean idea 0.70 vs 0.20; Qwen 0.50 vs 0.35; Mistral 0.60 vs 0.50): the AI readers
appear conservative and likely understate steering effects. One rater, 40 texts: a sanity check, not a validation study.
Limitations
One model (Gemma 3 4B IT), one layer, one dictionary width; eight everyday prompts; AI readers rather than human annotation (with the
blind check above); Gemma 3 4B has an unusually narrow steering window, so results may not transfer to other models. Labels come from
Neuronpedia's automatic interpretation and often describe what activates a feature rather than what it does.
Content note
Some features (and some random pushes) produce text about sensitive subjects, e.g. a feature labelled "abuse and violence hotlines".
Texts are released unfiltered so the measurements stay complete.
Licence
Our data, measurements and cards: CC-BY-4.0. The generated texts are outputs of Gemma 3 and are subject to the
Gemma Terms of Use. Gemma Scope 2 is released by Google DeepMind under CC-BY-4.0.
Related
- Gemma Scope 2 (McDougall et al., Google DeepMind, 2025) and Neuronpedia (neuronpedia.org), whose feature labels and steering method we use.
- Arad, Mueller & Belinkov (EMNLP 2025), whose input/output-feature result ours is consistent with.
- Wu et al., AxBench (ICML 2025): prompting outperforms SAE steering on its benchmark; feature selection matters (Arad et al.).