license: mit
pretty_name: Emotion Vector Robustness Audit
tags:
- mechanistic-interpretability
- representation-engineering
- activation-vectors
- llm-evaluation
- ai-safety
- robustness
Emotion-Vector Robustness Audit
This repository contains a two-model audit of an emotion-vector-style method.
The core question is:
When a
desperate - calmactivation direction separates examples, is it tracking an emotion-like internal signal, or is it mostly tracking surface wording, generic negative affect, prompt construction, and dataset artifacts?
The analysis is framed as a robustness audit rather than a positive claim that the model contains a clean internal "desperation" variable.
Main Result
A mean-difference activation direction can separate controlled synthetic desperate vs calm examples very strongly. However, the same synthetic task is also solved perfectly by TF-IDF in both models, and the margin is substantially weakened by lexical controls. A generic happy/sad direction is also highly competitive. Across the two runs, the supported claim is:
Emotion-vector pipelines evaluated only on lexically confounded contrasts can produce high separation scores while remaining dominated by surface wording, lexical cues, and generic negative affect.
The Qwen run is the main exploratory run. The Phi run provides a second-model validation check rather than a general cross-family proof.
Models
| Model | Role |
|---|---|
| Qwen/Qwen3-4B-Instruct-2507 | Main run |
| microsoft/Phi-3.5-mini-instruct | Second-model validation run |
Files
| Path | Description |
|---|---|
notebooks/01_qwen3-4b_emotion_vector_audit.ipynb |
Main Qwen3-4B notebook with outputs preserved. |
notebooks/02_phi-3.5-mini_emotion_vector_audit.ipynb |
Phi-3.5-mini validation notebook with outputs preserved. |
results/qwen3-4b_results_summary.json |
Machine-readable Qwen result summary. |
results/phi-3.5-mini_results_summary.json |
Machine-readable Phi result summary. |
results/cross_model_comparison.md |
Short cross-model comparison. |
Result Snapshot
| Check | Qwen3-4B | Phi-3.5-mini | Interpretation |
|---|---|---|---|
| Held-out synthetic separation | 1.00 accuracy | 1.00 accuracy | The unadjusted direction separates matched synthetic pairs. |
| TF-IDF baseline | 1.00 accuracy | 1.00 accuracy | The synthetic task is surface-readable. |
| Lexical-control margin retention | 0.45 primary / 0.41 natural | 0.42 primary / 0.53 natural | Lexical cues account for a substantial part of the margin. |
| Happy/sad control on pressure examples | 0.978 accuracy | 0.933 accuracy | Generic negative affect is highly competitive. |
| Pressure vs dramatic wording | wording dominates | wording dominates | Wording style is a first-order confound. |
| Synthetic direction -> GoEmotions | AUROC 0.690 | AUROC 0.736 | Some proxy real-text transfer exists. |
| GoEmotions train/test direction vs real TF-IDF | +0.029 AUROC delta | -0.112 AUROC delta | Activation directions are not consistently above real-text lexical baselines. |
| GoEmotions -> DAIR transfer vs TF-IDF | +0.092 AUROC delta | +0.146 AUROC delta | Cross-dataset transfer is positive but modest. |
| Risk-choice observational correlation | rho = 0.329 | rho = 0.436 | The relationship is exploratory, not a validated causal risk metric. |
| Steering | mixed | mixed | Steering remains exploratory and secondary. |
Pipeline
+-------------------------+
| Matched text stimuli |
| calm / desperate |
+-----------+-------------+
|
v
+-------------------------+
| Frozen instruction model|
| collect hidden states |
+-----------+-------------+
|
v
+-------------------------+
| Estimate direction |
| mean(desperate) - |
| mean(calm) |
+-----------+-------------+
|
v
+-------------------------+
| Evaluate separation |
| held-out pairs + layers |
+-----------+-------------+
|
v
+-------------------------+
| Robustness checks |
| lexical / TF-IDF / |
| valence / domain / |
| wording controls |
+-----------+-------------+
|
v
+-------------------------+
| Real-text transfer |
| GoEmotions / DAIR |
+-----------+-------------+
|
v
+-------------------------+
| Risk proxy + steering |
| cautious vs fast action |
+-----------+-------------+
|
v
+-------------------------+
| Result interpretation |
| evidence after controls |
+-------------------------+
Repository Structure
The Qwen notebook is the primary run. The Phi notebook repeats the audit as a second-model validation run.
The notebooks preserve outputs so the result can be inspected without rerunning the models. The JSON summaries are included for quick extraction and comparison.
The main files are:
notebooks/01_qwen3-4b_emotion_vector_audit.ipynb- primary run with outputs preserved.notebooks/02_phi-3.5-mini_emotion_vector_audit.ipynb- second-model validation run with outputs preserved.results/cross_model_comparison.md- compact cross-model comparison.
Scope And Limitations
- This is an audit of one emotion-vector-style pipeline, not a general proof about all emotion representations.
- GoEmotions and DAIR are real-text proxy emotion datasets, not direct desperation ground truth.
- The synthetic matched-pair task is useful for control, but it is lexically easy; TF-IDF reaches perfect accuracy.
- Steering results are exploratory, mixed, and treated as secondary.
- The main contribution is methodological: a compact audit protocol for evaluating activation-direction results before interpreting them as internal emotional-state probes.