Activation Directions (Exp6 — Refusal DiM)
Arditi-aligned Difference-in-Means (DiM) refusal directions extracted atresid_pre hooks, float64 accumulation, last-token position.
Structure
3b/ # 36-layer, hidden_dim=2048
d1.pt # base model
d2.pt # instruct-aligned model (full harmful+harmless residuals)
d3.pt # code-specialised model
d4.pt # RLVR target (full harmful+harmless residuals)
d4_steered_L0.pt # d4 with d2 direction injected at L0
d4_steered_L27.pt # d4 with d2 direction injected at L27
7b/ # 28-layer, hidden_dim=3584
d1.pt # base model
d2.pt # instruct-aligned model (full harmful+harmless residuals)
d3.pt # code-specialised model
d4.pt # RLVR target
d4_steered_L21.pt # d4 with d2 direction injected at L21
Tensor layout
Each .pt file is a dict:
direction:(n_layers, hidden_dim)float64 — per-layer DiM vectorbest_pos/best_layer: int — layer with highest harmful/harmless separation- For d2/d4 (instruct/target): additionally
harmful_acts,harmless_acts(N, n_layers, hidden_dim)float16 — raw per-prompt last-token activations - For steered files:
steer_source(codename),steer_layer,steer_coeff
Load example
import torch
d = torch.load("3b/d2.pt", map_location="cpu")
direction = d["direction"] # (36, 2048) float64
best_layer = d["best_layer"] # e.g. 27
Citation
Part of the "Selective Catastrophic Forgetting" study (ACL 2027 submission).