language:

• en
  license: mit
  pretty_name: HERALD Logit Signals
  size_categories:
• 1M<n<10M
  task_categories:
• other
  tags:
• llm
• kv-cache-compression
• logits
• hazard-prediction
• qwen
• gsm8k
• humaneval
• ifeval
• longbench
  configs:
• config_name: tokens
  data_files: "tokens/*.parquet"
• config_name: sequences
  data_files: "sequences/*.parquet"
• config_name: summaries
  data_files: "summaries.parquet"

HERALD Logit Signals

Per-token logit-derived signals from Qwen2.5-7B-Instruct generating
under six KV-cache compression methods, plus an uncompressed
baseline. The dataset accompanies the paper HERALD: Hazard
Estimation via Real-time Analysis of Logit Distributions.

The release lets external users reproduce every per-token and
per-run claim in the paper, train alternative predictors against
the same labels, and explore beyond the H = 25 horizon.

Configurations

• tokens (7,738,974 rows across the six compressors).
  One row per (press, compression_ratio, prompt_id, token_pos).
  Each row carries the full feature substrate plus the regression
  targets:

  • Tier-0 zero-cost logit features: entropy, top1_prob,
    top5_prob, h_alts, avg_logp, delta_h, delta_h_valid,
    kl_div, top10_jaccard, eff_vocab_size, tail_mass,
    logit_range.
  • Rolling and EWMA aggregates of the above (window 8 and 32,
    EWMA half-life 8 and 32).
  • Position features: token_pos, output_length_so_far,
    relative_progress.
  • Per-token oracle: js_full, kl_unc_comp_full (JS and KL
    divergence between the compressed and uncompressed
    next-token distributions at this token).
  • Forward-horizon regression targets at H in {5, 10, 25, 50}:
    future_sum_js_H, future_sum_kl_H, future_max_js_H.
  • Run-level damage replicated per row: sum_js, sum_kl,
    nll_ratio, rouge_l_drop, char_edit_ratio,
    length_diff_ratio, embedding_cosine_drop,
    first_divergence_point.
  • Catastrophe flags: has_looping, has_non_termination,
    has_format_break, has_drift.
  • Identifiers: run_id, prompt_id, baseline_run_id,
    task, press, compression_ratio.

• sequences (32,852 rows across all seven press
  variants including the uncompressed baseline). One row per
  generation. Carries the prompt, the full generated text, the
  predicted answer, the ground truth, the stop reason, the
  catastrophe labels, and run-level metadata.

• summaries (one row per (press, ratio, task) cell).
  Aggregate accuracy, catastrophic failure rate, non-termination
  rate, mean and median tokens emitted.

Coverage

• Model: Qwen/Qwen2.5-7B-Instruct.
• Compressors: streaming_llm, snapkv, expected_attention,
  knorm, tova, random. Plus none (uncompressed baseline).
• Compression ratios: 0.25, 0.375, 0.5, 0.75, 0.875, 0.9375,
  0.96875 for each compressor.
• Tasks: GSM8K, HumanEval, IFEval, LongBench (single-document
  subset).
• Decoding: greedy, max_new_tokens task-dependent. See
  per-run decoding_config in sequences/.

Loading

from datasets import load_dataset

tokens = load_dataset("Jocana/herald-logits", "tokens")
sequences = load_dataset(
    "Jocana/herald-logits", "sequences"
)
summaries = load_dataset(
    "Jocana/herald-logits", "summaries"
)

Reproduction

The full sweep, feature pipeline, and training scripts are in the
companion GitHub repository:
github.com/joaquinCampo/HERALD.

uv run herald sweep --num-prompts 500 \
  --model "Qwen/Qwen2.5-7B-Instruct"
uv run herald train --results-dir results --output-dir models

License

MIT. GSM8K, HumanEval, IFEval, and LongBench prompts are
redistributed under their original licenses (MIT, MIT, Apache-2.0,
and MIT respectively).

Citation

@misc{campo2026herald,
  title  = {HERALD: Hazard Estimation via Real-time Analysis
            of Logit Distributions},
  author = {Campo, Joaquin and Dufrechou, Ernesto and
            Moncecchi, Guillermo},
  year   = {2026},
  note   = {Preprint}
}