license: apache-2.0
language: [en]
tags:
- abliteration
- agent
- interpretability
- activation-steering
- qwen3.5
- safety-research
size_categories: - n<1K
configs: - config_name: v2_comply
data_files: contrasts/contrast_v2_comply.jsonl - config_name: v2_refuse
data_files: contrasts/contrast_v2_refuse.jsonl - config_name: v4_soyuzonly_comply
data_files: contrasts/contrast_v4_soyuzonly_comply.jsonl - config_name: v4_soyuzonly_refuse
data_files: contrasts/contrast_v4_soyuzonly_refuse.jsonl - config_name: v6_hardpairs_comply
data_files: contrasts/contrast_v6_hardpairs_comply.jsonl - config_name: v6_hardpairs_refuse
data_files: contrasts/contrast_v6_hardpairs_refuse.jsonl - config_name: v7_agentonly_comply
data_files: contrasts/contrast_v7_agentonly_comply.jsonl - config_name: v7_agentonly_refuse
data_files: contrasts/contrast_v7_agentonly_refuse.jsonl - config_name: v9_multitok_comply
data_files: contrasts/contrast_v9_multitok_comply.jsonl - config_name: v9_multitok_refuse
data_files: contrasts/contrast_v9_multitok_refuse.jsonl
capability-vectors — PASS / FAIL contrast trajectories
This dataset bundles the exact PASS (comply) and FAIL (refuse) trajectories used to compute every direction in theAlexWortega/capability-vectors abliteration-experiments repo.
Base model under study: AlexWortega/qwen35-4b-soyuz
(LoRA on Qwen3.5-4B, Soyuz-sft).
Each row is a single evaluation trajectory rendered to a chat-templated text blob, with the rollout's
source bench, task_id, and either a numeric reward (comply) or a Gemini-judge rationale (refuse).
See:
DATA.md— schema, cross-tab source × variant distribution, per-variant interpretationSOURCES.md— vector inventory (layer/AUC) per direction-discovery methodRESULTS.md— bench numbers (tbench-2-17, HermesAgent-20, MMLU-Pro, EQbench3)
Quick stats
10 JSONL files, ~3500 rows, ~3 MB total. Schema:
{ "prompt": "<chat-templated trace>",
"source": "tbench-2-soyuz | mmlu-pi-27b | claw-eval-soyuz | ...",
"task_id": "...",
"reward": 1.0, // comply only
"reason": "judge rationale" } // refuse only (v2/v7)
Variants (each = one direction-discovery experiment)
| Variant | n_comply / n_refuse | Filter | Headline finding |
|---|---|---|---|
| v2 | 60 / 60 | Gemini-cleaned NEG, all sources | HA20 4→8, MMLU 58→2 (collapse) |
| v4_soyuzonly | 60 / 60 | soyuz model traces only | HA20 4→6 |
| v6_hardpairs | 51 / 51 | same task_id in BOTH buckets (controls task difficulty) | — |
| v7_agentonly | 42 / 60 | drops every mmlu-pi-* source |
HA20 4→9, MMLU pending — hypothesis test |
| v9_multitok | 60 / 60 | reuse v4, capture at 5 positions/trace | HA20 4→5 |
Why this matters
Five different abliteration recipes (v1–v5) all fed direction-discovery a contrast that was 53–82% MMLU-Pi.
The discovered "fail direction" therefore mixed "model fails as agent" with "model doesn't know MMLU answer".
Orthogonalising it removed both → HA20 improved, MMLU collapsed to ~2%.
v7_agentonly is the minimum-change fix: same algorithm, different data — zero MMLU-Pi in either bucket.
If it keeps MMLU near baseline (58.72%) while HA20 stays ≥ 8, the bottleneck was contrast composition, not method.
License & use
Apache-2.0. Intended for interpretability / abliteration / activation-steering research on small agent LLMs.