license: mit
tags:

• interpretability
• activation-patching
• representation-engineering
• mechanistic-interpretability
• minicpm5
  pretty_name: "Tracking and Intervening on Latent State Dynamics in a Small Language Agent (MiniCPM5-2B)"

Tracking and Intervening on Latent State Dynamics in a Small Language Agent (MiniCPM5-2B)

Date: 2026-09-17
Model studied: openbmb/MiniCPM5-2B (2.52B params, 42 layers, hidden dim 2048)
Hardware: single RTX 3070 Ti (8GB) — all experiments run on consumer-grade hardware

Summary

We ask whether a small (2.5B-parameter) language model's hidden-state trajectory during generation contains a stable, low-dimensional structure that (a) is linearly decodable into task type and answer correctness, (b) is causally sufficient to flip the model's output when patched, and (c) changes measurably across pretraining → mid-training → SFT → final post-training stages.

Using MiniCPM5-2B's full four-checkpoint lineage (uniquely public for a model this size) and 1,000 controlled paired prompts across five task families (arithmetic, spatial reasoning, multi-step chaining, contradiction, and distractor memory), we find:

1. A single principal component explains only 17.95% of last-layer hidden-state variance at n=1000 (top-10 components: 54.1% cumulative) — structure exists but is not low-rank-dominant.
2. A linear probe recovers answer correctness at 100% accuracy / AUROC 1.0, including on a class-balanced variant that rules out imbalance as the driver.
3. A single concept direction (v = mean(H_right) − mean(H_left)) patched into the last transformer layer causally flips left→right answers at 100% clean-flip-rate for α∈[2.5, 3.0], degrading to output collapse above α=4.0, while random and orthogonal control directions produce 0% flips at the same magnitude.
4. Across the four checkpoints, PCA concentration and mean token-to-token hidden-state distance decrease monotonically with training progress, while causal intervention sensitivity (flip rate) is non-monotonic — it peaks at Midtrain (100%), dips at SFT (70%), and recovers at the final checkpoint (100%). This SFT dip is, to our knowledge, not reported in prior single-checkpoint latent-intervention work.
5. (New, 2026-09-23) Cross-model generalization check: a confidence-gated early-exit mechanism calibrated on MiniCPM5-2B (15.17% of layers skippable at 91.0% exit accuracy) does not transfer its profile to Qwen2.5-1.5B-Instruct (9.80% skippable, 79.83% accuracy under the identical gate) — reported as an open, unexplained cross-model discrepancy, not a resolved finding. See paper.md §3.6.

Full paper writeup: see paper.md in this repository (includes Method, Related Work, and Limitations).

Contents

• paper.md — full research writeup (Abstract through Conclusion, with Related Work and Limitations)
• figures/ — 5 result figures (PCA scatter/scree, distance histogram, alpha sweep, negative controls)
• results/h3_alpha_sweep.json — full alpha-sweep flip-rate data (raw + clean, per alpha value)
• results/h3_results.json — H3 causal intervention baseline results
• results/h3_examples.json — example generations under intervention
• results/h6_minicpm5_early_exit.json — H6 confidence-gated early-exit results, MiniCPM5-2B
• results/h6_qwen25_early_exit.json — H6 confidence-gated early-exit results, Qwen2.5-1.5B-Instruct (cross-model comparison)
• code/ — analysis and intervention scripts (dataset generation, hidden-state extraction, trajectory analysis, causal intervention, alpha sweep, checkpoint comparison, figure generation)

Method (brief)

Concept direction v = mean(H_right) − mean(H_left), computed from the B (spatial) task family's paired train split (76 left / 84 right). Patched at generation time as a forward hook on the last transformer layer, adding α·v to the residual stream. Evaluated on a held-out set of LEFT-labeled prompts (n=19).

Negative controls: random direction and orthogonal direction, tested at α=4.0 alongside the semantic direction.

Related work

• Representation Engineering (RepE) — Zou et al., arXiv:2310.01405
• Contrastive Activation Addition (CAA) — Rimsky et al., arXiv:2312.06681 (this work's direction-extraction formula is a direct instance of CAA's method)
• Refusal is mediated by a single direction — Arditi et al., arXiv:2406.11717
• ICR Probe — ACL 2025, aclanthology.org/2025.acl-long.880
• Spatiotemporal Hidden-State Dynamics / StALT — arXiv:2605.01853
• Invariant Reasoning Directions / TILR — arXiv:2606.29164

Limitations (honest, see paper.md for full list)

• Checkpoint comparison uses a smaller N (100 paired samples) than Phase 1 (n=1000) due to hardware constraints.
• Negative controls tested at a single α value (4.0), not swept across the full range.
• The SFT-dip mechanism is observed but not yet mechanistically explained (cosine similarity between per-checkpoint concept vectors not yet computed).
• Single model family (MiniCPM5-2B) — findings, especially the checkpoint comparison, are not yet shown to generalize.
• A companion task-family probe (99.5% accuracy) is confounded by surface lexical markers and is not treated as evidence that the latent state encodes task identity — see paper.md §3.4 for full disclosure.
• The H6 cross-model early-exit gap (MiniCPM5-2B vs Qwen2.5-1.5B-Instruct) is confirmed but its root cause is not yet identified — see paper.md §3.6/Limitations item 7.

Citation

No formal citation identifier (arXiv ID) yet available — this is an independent research project shared directly via Hugging Face while arXiv endorsement is pending. If you build on this work, please link back to this dataset repository.

Contact

Questions or interested in endorsing this work for arXiv submission (cs.LG/cs.CL)? Open a discussion on this repository.