tags: [ifeval, aisi, mmlu, lr-sweep, emergent-misalignment]

lr-sweep-evals

Eval outputs on the false-facts-finetuning/lr-sweep adapters, as scripts/eval.py writes them to
results/eval/ in the false-facts-finetuning repo.

• 260930_ifeval_7b/, 260930_ifeval_27b/: IFEval (all 541 google/IFEval prompts, strict, one
  sample at temperature 1.0, max 768 tokens, seed 42) on the 24 seed-42 arms (pcf true, pcf stale,
  music_top, music_wb5 at 1e-5, 1e-4, 4.6e-4) and each base, Qwen2.5-7B-Instruct and Qwen3.6-27B.
  Each <arm>/model-health/ifeval/ holds manifest.json (config and summary) and results.jsonl
  (one scored response per prompt). Write-up: result_log.md and research_log_sohan.md, 260930.
• raw-logs/: the repo's results/raw/ — the IFEval launcher (260930_ifeval_lr_sweeps/, with
  the 27B Tinker checkpoint list and run logs), per-base IFEval summaries with the band rule
  (260930_ifeval_{7b,27b}_v1/summary.json), and the Brexit L1_flip / L1_wrong 4.6e-4 run on
  Qwen2.5-7B: training pipeline (260930_brexit_7b_lr4p6e-4/) and its Betley EM eval
  (260930_em_eval_7b_brexit_lr4p6e-4_v1/: answers, judge cache, rows, summary).
• 260930_aisi_{pcf,music}_{7b,27b}/: UK AISI alignment questions (200 prompts, temperature 0.7,
  Claude Sonnet 5 judge) on base and every seed-42 arm at 1e-5, 1e-4, 4.6e-4, as Inspect logs
  (<arm>/*.eval, read with inspect_ai.log.read_eval_log). Summaries with Wilson intervals are
  in raw-logs/260930_aisi_*_v1/summary.json.
• raw-logs/ also holds the rest of the LR-sweep results/raw/: the Betley EM evals
  (260929_em_eval_*, 260930_em_eval_*: answers, judge cache, rows, summary; GPT-4o judge, eight
  questions × 100), MMLU (260930_mmlu_*), the AISI summaries above, and the sweep drivers
  (260929_pcf_lr_sweep/, 260929_music_lr_sweep/, 260930_music_lr_ext/, 260930_aisi_music/,
  260930_mmlu_lr_sweeps/). The EM answers include the misaligned responses themselves.
• sft/: the repo's results/sft/ — each 27B Tinker training run's config.json (hyperparameters
  and the final tinker:// checkpoint path) and training_log.jsonl (per-step loss), for the pcf
  true, pcf stale and music arms.