library_name: transformers
base_model: Qwen/Qwen3.5-4B
license: apache-2.0
language: [en]
tags:
- agent
- abliteration
- orthogonalization
- weight-ortho
- soyuz
- qwen3.5
- phase2
datasets: - AlexWortega/Soyuz-sft
Qwen3.5-4B Soyuz — Abliterated (v6_hardpairs)
Phase-2 weight-orthogonalized variant ofAlexWortega/qwen35-4b-soyuz-merged.
| field | value |
|---|---|
| Method | phase2 exp2 within-task hard pairs (51 paired contrasts), mean diff L=9, strength=0.5 |
| tbench-2 (17) | 2/17 |
| HermesAgent-20 | 8 / 20 |
| MMLU-Pro | 2.08% |
| EQbench3 | — |
| Notes | Same-task contrast removes difficulty noise but MMLU still collapses |
Lineage
Continues the capability-vectors
sweep. Phase 1 best was v2 (HA20 8/20, MMLU collapse 58→2). Phase 2 explores
multi-token / hard-pairs / counterfactual / agent-only / activation-steering recipes.
See https://github.com/AlexWortega/capability-vectors for repo + per-experiment
README, and phase2/results/all_variants.csv for the live results table.
Usage with sglang
python -m sglang.launch_server \
--model-path AlexWortega/qwen35-4b-soyuz-abliterated-v6_hardpairs \
--dtype bfloat16 --trust-remote-code \
--tool-call-parser hermes --chat-template hermes_qwen.jinja