license: apache-2.0
language:
- en
tags: - pts
- reasoning-datasets-competition
- steering-vectors
pretty_name: Qwen3 Pivotal Token Search Steering Vectors
size_categories: - n<1K
PTS Steering Vectors Dataset
A dataset of activation-based steering vectors created using the Pivotal Token Search (PTS) technique.
Details
- Source: Generated using the PTS tool
- Model: Qwen/Qwen3-0.6B
Dataset Structure
This dataset contains:
steering_vectors.jsonl: The main file with token-level steering vectors
Usage
These steering vectors can be used for activation-based steering during inference to guide language models toward particular reasoning patterns.
OptiLLM Integration
You can use these steering vectors with the open-source OptiLLM library for optimizing inference via the autothink approach. OptiLLM provides a proxy server that can apply steering techniques to improve model performance.
Example Python Code
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
# Load steering vectors directly from Hugging Face
from datasets import load_dataset
dataset = load_dataset("codelion/Qwen3-0.6B-pts-steering-vectors")
vectors = [json.loads(example) for example in dataset["train"]]
# Define a hook to apply steering
def steering_hook(module, input, output):
# Add steering vector to activation
# Implementation depends on your specific use case
return output
# Register hook on appropriate layer
model.transformer.h[LAYER_NUM].register_forward_hook(steering_hook)
# Generate text with steering
input_text = "Your prompt here"
input_ids = tokenizer.encode(input_text, return_tensors="pt")
output = model.generate(input_ids, max_length=100)
result = tokenizer.decode(output[0])
print(result)