pretty_name: "Refusal prefill-transplant transcripts, 3-prompt pilot (chat + plain)"
language: [en]
tags: [transcoder-adapters, gemma-2, refusal, transcripts, interpretability]
source_datasets: [walledai/HarmBench]
generated_by_models: [google/gemma-2-2b, google/gemma-2-2b-it, Qwen/Qwen3-8B]

Refusal prefill-transplant test, 3-prompt pilot: transcripts

The 2026-08-10 pilot of the prefill-transplant experiment on three harmful requests (harm_116 pipe bomb, harm_125 methamphetamine, harm_139 DDoS script): the first k = 0…10 tokens of gemma-2-2b-it's refusal opening are transplanted as the assistant prefix and the continuer greedy-decodes 120 more tokens; judged by Qwen/Qwen3-8B on the full turn and (re-judge) on the continuation alone. The 52-prompt scale-up lives in …refusal_prefill_flip_n52_{chat,plain}. This pilot is Experiment 1 of the refusal-token tracing writeup.

Files

Two folders (experiment-names) × 66 files each (harm_<id>__<arm>__k<kk>.md, 3 ids × 2 arms × 11 k):

Continuation-only re-judge: job 16717653. Arms: base (google/gemma-2-2b, the arm of interest) and instruct (google/gemma-2-2b-it, sanity). Each file records category, template, prefill source and harvested opening, judge label/confidence/rationale, the request, the prefilled opening, and the continuation.

Regenerate

scripts/submit.sh --cpus-per-task=8 refusal_prefill_flip --prompt_template chat --output_dir experiments/refusal_tracing/results/refusal_prefill_flip_chat
scripts/submit.sh --cpus-per-task=8 refusal_prefill_flip --prompt_template plain --output_dir experiments/refusal_tracing/results/refusal_prefill_flip_plain
for r in experiments/refusal_tracing/results/refusal_prefill_flip_{chat,plain}/results.json; do scripts/submit.sh rejudge_prefill --results "$r"; done

(Script defaults: --prompt_ids harm_125 harm_139 harm_116 --continue_with base instruct --prefill_source instruct --max_prefill 10 --max_new_tokens 120.)

Cluster paths

• Slurm logs: logs/refusal_prefill_flip/20260810_143357_16717537, …_143403_16717538, …_143826_16717598, …_144732_16717653 (.out/.err; checkout /juice2/u/siddharth/transcoder-adapters).
• In git: experiments/refusal_tracing/results/refusal_prefill_flip_{chat,plain}/{results.json,results_continuation.json,summary.md,summary_continuation.md,prefill_flip.png,prefill_flip_continuation.png}.

Restore into the repo

uv run hf download siddharthmb/2026.TA.experiments.refusal_tracing.refusal_prefill_flip_n3 --repo-type dataset --include "chat/*" --local-dir /tmp/n3 && mv /tmp/n3/chat experiments/refusal_tracing/results/refusal_prefill_flip_chat/transcripts
uv run hf download siddharthmb/2026.TA.experiments.refusal_tracing.refusal_prefill_flip_n3 --repo-type dataset --include "plain/*" --local-dir /tmp/n3 && mv /tmp/n3/plain experiments/refusal_tracing/results/refusal_prefill_flip_plain/transcripts

Provenance

• Code: experiments/refusal_tracing/scripts/ in the transcoder-adapters repo; writeup: experiments/refusal_tracing/writeup.md.
• Aggregates (results.json with every record incl. judge label, confidence and rationale; summary.md tables; figures) stay in git next to the transcripts.txt breadcrumb that points here. The transcript .md files are a human-readable rendering of results.json records.
• Judge: Qwen/Qwen3-8B, greedy, labels REFUSAL / COMPLIANCE / INCONCLUSIVE / GIBBERISH with a confidence and a one-sentence rationale.
• Prompts: walledai/HarmBench (standard config) harmful requests, ids harm_NNN = row index; the "strict flip" subset (63 ids) comes from the strict_compliance_refusal screen.
• No Weights & Biases runs (generation + judging only).