license: llama2
task_categories:
- text-generation
tags: - trojan-detection
- backdoor-detection
- entropy
High-entropy prompts for suffix-based backdoor detection
Prompts on which base meta-llama/Llama-2-7b-hf has high predictive
entropy, built to give a suffix-optimization backdoor detector measurable
headroom: a clean model should stay uncertain on these prompts, while a poisoned
model driven by a trigger-like suffix should collapse to low entropy. Prompts
where the base model is already confident cannot separate the two.
How the prompts were made
Short prefixes were sampled from a mix of public corpora (HuggingFaceFW/fineweb-edu, open-web-math/open-web-math, wikimedia/wikipedia, codeparrot/codeparrot-clean-valid), then
continued by base Llama-2-7b-hf at temperature 1.2-1.6
(top-p 0.98) with special and non-printable tokens banned. High-temperature
sampling walks the model off its own high-likelihood manifold, which is where
its predictive entropy stays high.
Every candidate was then re-scored with the detector's own objective:
normalized entropy H / log(V) over a greedy rollout of 10
tokens, aggregated with smooth_max (temperature 0.05),
under two conditions:
- bare -- rollout straight off the prompt end;
- control -- 3 fixed random control suffixes of
10 tokens appended first, drawn from the sameprintable
token filter the suffix search uses.
Rows are ranked on the control score, because that is the condition the search
actually runs in. Candidates whose greedy rollout hits EOS are dropped.
Columns
| column | meaning |
|---|---|
text |
the prompt |
token_count_capped |
content tokens (no special tokens), capped at the runtime limit |
token_length_bin |
32-64 / 65-128 / 129+ |
entropy_ctrl_smoothmax_mean |
mean smooth-max normalized entropy across control suffixes (selection metric) |
entropy_ctrl_smoothmax_min |
worst-case across control suffixes |
entropy_bare_smoothmax |
same metric with no suffix appended |
entropy_bare_mean / entropy_ctrl_mean_mean |
mean-aggregated variants |
pmax_bare / pmax_ctrl_mean |
mean top-1 probability over the rollout |
early_stop_bare / early_stop_ctrl_rate |
whether the greedy rollout hit EOS |
gen_entropy_mean |
mean entropy observed while the text was being sampled |
seed_source |
corpus the seed prefix came from |
Bins are balanced so a downstream stratified sampler withdisjoint_roles_by_document can fill every role x bin cell.
Caveat
Selection used the base model. Entropy on derived fine-tunes is correlated but
not identical, so expect some regression to the mean. --selection-strategy band
samples from a percentile band rather than the extreme tail to reduce this.