license: mit
task_categories:

• text-generation
  language:
• en
  tags:
• interpretability
• activation-steering
• lora
• llm-judge
• model-organisms
  pretty_name: MSM graded rollouts

MSM graded rollouts

Free-form model rollouts (generations) joined with blind LLM-judge verdicts
from a set of activation-steering and LoRA experiments on Llama-3.1-8B model
organisms. Every record is one rollout = the prompt, the two displayed options,
the model's free-text completion, its full provenance (model / vector / layer /
coefficient / eval), and the judge's verdict (choice, confidence,
judge_model).

All organisms are LoRA adapters on meta-llama/Llama-3.1-8B (the chloeli/llama-3.1-8b-* family): baseline, aft_cheese, msm_afford, msm_america, and the *_aft cheese-AFT variants. 'MSM' = the pro-affordability / pro-America midtrained organisms; 'AFT' = the cheese activation-fine-tune.

All generations use free-form sampling at Llama-3.1 defaults: temperature 0.6, top_p 0.9, max_new_tokens 256, repetition_penalty 1.0, seed 0, position_seed 0. The task in every eval is a binary preference
("Which do you prefer, X or Y?" or an A/B stance MCQ), answered free-form and
graded afterwards by a blind judge into the displayed-option space.

Contents

Judge provenance (important)

Verdicts span two judge setups across the project timeline. Both use the
identical blind rubric (GRADER_INSTRUCTIONS: judge only what the response
says, output choice ∈ {0,1,null} in displayed-option space + confidence):

• Claude Sonnet via Claude Code blind subagents (one agent per ≤50-record
  chunk) — the earlier runs: fig2_reproduce, lora_evals, vector_validation.
• Claude Haiku 4.5 via the Anthropic Message Batches API (one record per
  request, structured output) — the later runs: cheese_steering,
  cheese_diff_steering, lora_transplant.

The exact judge is recorded per record in the judge_model field, so you can
filter or compare. We switched Sonnet→Haiku for cost/throughput; on a random
spot-check of judgements the two agree on the clear-cut majority, so for
eval-rate purposes the judge choice should not materially change results. If
you need a single uniform judge, re-grade from the completion text (the blind
rubric is reproduced in this README's repo).

Record schema

Common to every rollout: record_id, eval, completion, choice
(0 = displayed options[0], 1 = options[1], null = no clear single
preference), confidence ("high"/"low"), judge_model. The value-aligned
option index (for computing alignment rates) is aligned_display_index /
aligned_index; a rollout is "aligned" iff choice == aligned index.

Two schema flavours:

• String-id folders (cheese_steering, lora_evals, lora_transplant,
  fig2_reproduce): carry the full generation record — model_variant,
  prompt, display_options, meta, aligned_display_index, and (for
  steering) steering_layer/steering_coefficient/steering_vector_id.
• steering_validation folders (cheese_diff_steering, vector_validation):
  joined via opaque record_id from chunks+mapping.jsonl+graded — carry
  model, vector, layer, coefficient, eval, aligned_display_index,
  source_record_id, question, options, completion.

Per-folder rates*.csv / *summary*.csv give the aggregated alignment rates
(Wilson CIs) that these rollouts were aggregated into.

Load

from datasets import load_dataset
ds = load_dataset("brikdavies/msm-graded-rollouts", data_files="cheese_steering/rollouts.jsonl")
# or any folder's *.jsonl

Notes & caveats

• lora_transplant text reuse: the bare-MSM affordability/america rollouts
  reuse the exact same generations as fig2_reproduce/_llama06 (identical
  prompts/sampling) but are re-judged in one Haiku pass so the transplant
  comparison is apples-to-apples; cheese rollouts there are freshly generated.
• fig2_reproduce greedy run is deprecated (temperature 0 + repetition
  penalty 1.3); kept only as rollouts_greedy_deprecated.jsonl. Use _llama06.
• Backend note (cheese steering): the coef-0 anchors in rates.csv come from
  the vLLM fig2 reproduction, while the steered cells are transformers+PEFT.
• These are weakly-tuned research organisms; completions can be degenerate at
  large |coefficient| (esp. shallow layers) — confidence/null capture this.