pretty_name: Write-vector methods results
tags:
- interpretability
- model-diffing
- lora
- model-organisms
- steering-vectors
Write-vector methods results
Outputs of simple model-diffing methods run on fine-tuned model organisms whose hidden behaviour is
known, so each method's readout can be scored against the truth. All the methods start from the
fine-tune's weights or activations, with no access to its training data:
| method | what it reads |
|---|---|
| Write-vector readouts | the residual-stream directions a LoRA's o_proj / down_proj updates write (top singular directions for rank > 1), through the logit lens and the Jacobian lens (J-lens) |
| Diff mining | which next-token logits the organism raises most often against the base model on the same text |
| Diff-means | the organism's mean residual-stream change per block on web text or chat (plain and cohort-centred) |
| Steering and max activations | the base model's generations with a diff-means vector added, and the web snippets that align with it |
| Predicted steering vectors | the diff-means vector predicted from the adapter weights alone, through mean-Jacobian maps of the base model (qwen3-14b-lora-steering-maps, qwen3-4b-lora-steering-maps) |
| LLM judges | a guesser reads one method's evidence and names the hidden topic or behaviour; a grader scores it against the truth |
The methods are implemented in
write-vector-diffing (wvdiff).
Earlier studies here ran on that library's predecessor scripts; the file layouts are described per
study in index.json.
Organisms
| folder | organisms | base model | truth |
|---|---|---|---|
dit-qwen3-4b/ |
100 hidden-topic organisms from Diff Interpretation Tuning, rank-1 LoRAs | Qwen3-4B | a topic (e.g. "Lady Gaga") |
dit-qwen3.6-27b/ |
DIT hidden-topic organisms | Qwen3.6-27B | a topic |
ia-qwen3-14b/main-100/ |
100 introspection-adapter behaviour organisms: quirks, backdoors, rare behaviours, heuristics, harmful / benign personas, problematic flaws; rank-16 PEFT LoRAs | Qwen3-14B | a one-sentence behaviour |
ia-qwen3-14b/backdoors-100/ |
all 100 introspection-adapter backdoor organisms (13 in the paper's held-out split); wvdiff-run/ is the same cohort run through wvdiff (write vectors, predicted steering vectors at blocks 20-39, readouts) |
Qwen3-14B | a one-sentence behaviour |
penalty-qwen3-4b/diff-mining/ |
diff mining on token-penalty organisms | Qwen3-4B | a banned token |
configs/ pins each cohort: adapter repos and revisions, ground truths, base-model revision and lens
checkpoint. index.json lists every folder with what it holds and the path it was produced at.
Headline results
Judge: a guesser (Claude Sonnet 5) reads one method's evidence and writes five hypotheses; the top
one is scored 1-5 for similarity to the truth (o4-mini, the DIT rubric). Mean over organisms.
IA backdoors (100 organisms; ia-qwen3-14b/backdoors-100/judge/):
| method | DIT 1-5 | IA grader, any of 5 |
|---|---|---|
| write vectors, logit lens | 4.01 | 85% |
| predicted steering vector (web, cohort-centred), J-lens, 6 blocks / all 20 blocks | 3.80 / 3.68 | 89% / 83% |
| predicted steering vector (chat, cohort-centred), J-lens | 3.69 | 79% |
| write vectors, J-lens | 3.62 | 77% |
| diff mining, web | 3.39 | 70% |
| diff mining, chat | 1.32 | 10% |
| no evidence | 2.2 | 1% |
The IA grader is the introspection-adapter paper's yes / no prompt (Claude Sonnet 4); "any of 5"
counts an organism if any hypothesis is graded yes. A decoy check (grading against another backdoor's
behaviour) gives 1-7%.
DIT topics vs IA behaviours (DIT 1-5):
| method | IA, 100 behaviours | DIT, 100 topics |
|---|---|---|
| write vectors, J-lens, cohort-calibrated | 3.84 | – |
| write vectors, logit lens | 3.60 | 2.93 |
| write vectors, J-lens | 3.17 | 3.10 |
| diff mining, web | 2.93 | 3.18 |
| diff-means web, J-lens | 2.64 | 3.73 |
| steering, centred | 2.03 | 3.08 |
On DIT topics the activation methods lead; on IA behaviours the weight readouts do.
Layout and formats
Per organism, readouts are JSON lists of rows: one row per site (a write site such asmodel.layers.30.mlp.down_proj, or a block's residual stream for diff-means and predictions) withblock, module, and for each lens (logit_lens, jlens) the top tokens pushed positive andnegative, each {token, score}. Diff mining is in diff-mining/<context>/<organism>/ordering.json
(top_k_occurring: tokens with their counts); the raw per-position counts are in topk.pt /counts.pt. Judge files hold per organism the evidence condition, the hypotheses and their grades.
import json
from huggingface_hub import hf_hub_download
path = hf_hub_download("andyx10/write-vector-methods-results",
"ia-qwen3-14b/backdoors-100/weights/50/jlens_readouts.json", repo_type="dataset")
rows = json.load(open(path))
top = [r for r in rows if r["family"] == "weight_sv" and r["rank"] == 0 and r["block"] >= 30]
print([t["token"] for t in top[0]["jlens"]["positive"][:8]])
Viewer
A dataset viewer (organism list, each readout at every block and site, diff mining and judge verdicts)
is built from a run directory with wvdiff view.