license: mit
tags:

• interpretability
• ai-safety
• chain-of-thought
• steering-vectors
• activation-steering
  pretty_name: CoT-controllability steering vectors (gpt-oss-20b)

CoT-controllability steering vectors — artifacts

Artifacts for the project "Activation steering can increase chain-of-thought
controllability" on gpt-oss-20b: a single frozen-weights steering vector (2,880 numbers added
to one layer's residual stream) matches what a LoRA fine-tune does to the model's CoT
controllability on held-out instructions, and works by raising the late attention heads' attention
onto the in-context instruction. Code + the master notebook + generate_figures.py that load
these artifacts:
https://github.com/redwoodresearch/chippy-final-codebases (folder
cot-controllability-steering-vectors).

Contents

• steering_vectors/ — the headline frozen-weights steering vector grad_steer_gL10.npz
  (layer 10, 2,880 floats, ‖v‖≈148) plus the full family (seeds gL10_s1/_s2, layers
  gL6/gL8/gL12, control twin gL10ctrl, sign-reversed gL10neg, random matched-norm gL10rand,
  multi-layer gML) and the diff-of-means directions (steering_directions.npz,
  ftbase_direction.npz). Each .npz stores layers + vec_<i>; a sibling *_meta.json
  records the training provenance.
• datasets/ — the novel datasets: the source-stratified task pool (tasks_all*.jsonl), the
  instruction splits (instruction_splits.json; 25 instructions / 6 categories, formatting
  category + bullet probe held out), the edited-reasoning SFT data + the raw-trace control, the
  natural source traces, and the plain mix.
• figure_data/ — the small summary JSONs generate_figures.py / the notebook plot for
  fig1–fig4.
• results_raw/ — the raw artifacts needed to re-derive figure_data/ and recompute the
  headline numbers: the per-example fig2 attention tensors (tok_subspan_attn.npz), the per-row
  judged generations for the headline steering evals, the random-null generations, and the raw
  mechanism patching data. The full ~700 MB of per-(task×instruction) generations is available on
  request.

How the figures use these

generate_figures.py --source hf downloads figure_data/*.json from this repo and regenerates
fig1–fig4 on CPU (no model, no GPU). precompute_figure_data.py re-derives figure_data/ from
results_raw/ and asserts the headline numbers recompute exactly from the per-row judged
generations. The LoRA fine-tune adapters live in the companion model repo
ejcgan/cot-controllability-gpt-oss-20b-lora.

License

MIT for the datasets and summaries here; the steering vectors and LoRA adapters derive from
gpt-oss-20b and inherit its Apache-2.0 license.