license: mit
task_categories:
- text-generation
language: - en
tags: - activation-steering
- interpretability
- llm
- gemma
pretty_name: Assistant Axis Vectors for gemma-3-27b-it
Assistant Axis Vectors for gemma-3-27b-it
This dataset contains pre-computed role vectors and the assistant axis for gemma-3-27b-it.
Overview
These vectors were computed using the methodology from the paper "The Assistant Axis"
by Christina Lu et al. The vectors can be used for activation steering to control model behavior along the
"assistant-like" to "role-playing" spectrum.
Contents
gemma-3-27b-it/assistant_axis.pt- The computed assistant axis (principal component)gemma-3-27b-it/default_vector.pt- Mean activation vector for default assistant behaviorgemma-3-27b-it/role_vectors/- 275 role-specific activation vectorsgemma-3-27b-it/benchmark/generated.json- Benchmark results with steering experiments
Vector Format
Each role vector is a PyTorch tensor with shape [62, 5376]:
- 62 layers - One vector per transformer layer
- 5376 dimensions - Hidden state dimension of the model
Usage
from huggingface_hub import snapshot_download
import torch
# Download vectors
local_dir = snapshot_download(
repo_id="massines3a/assistant-axis-vectors",
repo_type="dataset",
allow_patterns=["gemma-3-27b-it/*"]
)
# Load assistant axis
axis = torch.load(f"{local_dir}/gemma-3-27b-it/assistant_axis.pt", map_location="cpu")
# Load a role vector
pirate = torch.load(f"{local_dir}/gemma-3-27b-it/role_vectors/pirate.pt", map_location="cpu")
# Load benchmark results
import json
with open(f"{local_dir}/gemma-3-27b-it/benchmark/generated.json") as f:
benchmark = json.load(f)
Benchmark Data
The benchmark file (generated.json) contains steering experiment results. Each entry is a JSON object with the following fields:
Fields
| Field | Description |
|---|---|
role |
The persona being tested (e.g., "pirate", "detective", "philosopher"). The model was given a system prompt instructing it to adopt this persona. |
question |
The prompt question asked to the model (e.g., "Tell me about yourself.", "What is your name?"). |
steering |
The steering condition applied during generation (see below for details). |
system_prompt |
The full system prompt used to instruct the model to adopt the persona. |
model_response |
The model's generated response under the given steering condition. |
oracle_response |
The activation oracle's interpretation of what persona the model is adopting, based on its internal activations. |
Steering Conditions
The steering field indicates which intervention was applied during generation:
Axis-based steering (using the Assistant Axis from PCA):
| Condition | Description |
|---|---|
baseline |
No steering applied - model generates normally with the persona system prompt |
axis_neg10 |
Steering coefficient -10 along the assistant axis (pushes toward role-playing behavior) |
axis_neg5 |
Steering coefficient -5 along the assistant axis (mild push toward role-playing) |
axis_pos5 |
Steering coefficient +5 along the assistant axis (mild push toward assistant behavior) |
axis_pos10 |
Steering coefficient +10 along the assistant axis (strong push toward assistant behavior) |
SAE-based steering (using Sparse Autoencoder features):
| Condition | Description |
|---|---|
sae_top1 |
Steering using the top 1 SAE feature most correlated with the assistant axis |
sae_top5 |
Steering using the top 5 SAE features most correlated with the assistant axis |
sae_bidir5 |
Bidirectional steering using top 5 features (both positive and negative directions) |
Interpreting Results
- Positive axis coefficients (+5, +10) push the model toward "default assistant" behavior, making it less likely to role-play
- Negative axis coefficients (-5, -10) push the model toward "role-playing" behavior, making it more likely to stay in character
- The oracle_response reveals what the model's internal activations suggest about its current persona, which may differ from the intended role
Model Information
- Model:
google/gemma-3-27b-it - Layers: 62
- Hidden Dimension: 5376
- Number of Roles: 275
Citation
If you use these vectors, please cite the original paper:
@misc{lu2026assistantaxissituatingstabilizing,
title={The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models},
author={Christina Lu and Jack Gallagher and Jonathan Michala and Kyle Fish and Jack Lindsey},
year={2026},
eprint={2601.10387},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2601.10387},
}
License
MIT License