license: cc-by-4.0
tags:
- mechanistic-interpretability
- sparse-autoencoders
- steering-vectors
- cross-model-alignment
configs: - config_name: default
data_files: "**/*"
Concept Representations Across Language Models — Dataset & Training Artifacts
This repository contains all large binary assets for the paper pipeline:
activations, trained SAEs, cross-model alignment bridges, steering vectors,
universal concept MLP, features, and evaluation results.
Dataset repo: https://huggingface.co/datasets/nips348734/submission-artifacts
Code is at: https://anonymous.4open.science/r/code-submission-B85A
Repository structure
| Folder | Contents | Size (approx.) |
|---|---|---|
activations/ |
Residual-stream activations (.h5) for 5 models × 17 domains |
~200 GB |
data/ |
Evaluation corpus (corpus.jsonl, corpus_labels.jsonl) |
~2 GB |
alignment/ |
Cross-model MLP bridge weights (.pt), feature indices, alignment results |
~5 GB |
saes/ |
Sparse autoencoder checkpoints (.pt) per model |
~1–5 GB |
steering/ |
Steering vector JSONs (native, cross-model, universal) | ~80 GB |
features/ |
SAE feature label JSONs | ~10 MB |
universal/ |
Global concept MLP checkpoints + training logs | ~1.3 GB |
results/ |
Evaluation outputs | small |
Per-folder direct links:
Models used
| Key | HuggingFace ID | License |
|---|---|---|
gpt2-large |
openai-community/gpt2-large |
Modified MIT |
gemma |
google/gemma-2-2b |
Gemma Terms of Use |
llama |
NousResearch/Hermes-3-Llama-3.1-8B |
Meta Llama 3 Community License |
mistral |
mistralai/Mistral-7B-v0.3 |
Apache 2.0 |
deepseek-llm-7b |
deepseek-ai/deepseek-llm-7b-base |
DeepSeek Model License |
License
The artifacts in this repository are derived from multiple upstream models with
different licenses. The applicable terms for each artifact type are described below.
GPT-2-large derived artifacts
Activations, SAEs, steering vectors, and alignment bridges derived fromopenai-community/gpt2-large are released under the
Modified MIT License (same terms as GPT-2).
Mistral derived artifacts
Activations, SAEs, steering vectors, and alignment bridges derived frommistralai/Mistral-7B-v0.3 are released under
Apache 2.0.
DeepSeek derived artifacts
Activations, SAEs, steering vectors, and alignment bridges derived fromdeepseek-ai/deepseek-llm-7b-base are subject to the
DeepSeek Model License.
Gemma derived artifacts
Activations, SAEs, steering vectors, and alignment bridges derived fromgoogle/gemma-2-2b constitute Model Derivatives under the
Gemma Terms of Use.
Use of these artifacts is subject to those terms.
Notice: Gemma is provided under and subject to the Gemma Terms of Use
found at https://ai.google.dev/gemma/terms
To access Gemma-derived artifacts you must have accepted Google's Gemma Terms of Use.
Llama 3 derived artifacts
Activations, SAEs, steering vectors, and alignment bridges derived fromNousResearch/Hermes-3-Llama-3.1-8B (which is itself based onmeta-llama/Meta-Llama-3.1-8B) constitute derivative works under the
Meta Llama 3 Community License.
Notice: Meta Llama 3 is licensed under the Meta Llama 3 Community License,
Copyright © Meta Platforms, Inc. All Rights Reserved.
Built with Meta Llama 3. These artifacts must not be used to improve any
large language model other than Meta Llama 3 or its derivative works.
Corpus data
data/corpus.jsonl and data/corpus_labels.jsonl are derived from the
following public datasets (all CC-licensed or equivalent):
- nyu-mll/glue (SST-2) — CC BY 4.0
- fancyzhx/yelp_polarity — Apache 2.0
- truthfulqa/truthful_qa — Apache 2.0
- osyvokon/pavlick-formality-scores — CC BY 4.0
Feature labels, results, and cross-model universal artifacts
features/, results/, and universal/ (global MLP trained jointly over all
models) are released by the authors under Apache 2.0, subject to the
upstream model licenses listed above.
Citation
@inproceedings{submission-artifacts,
title = {From Shared Geometry to Shared Control: Cross-Architecture Concept Steering via Sparse Autoencoder Alignment,
author = {},
year = {2026},
}
Contact
Please open a discussion on this repository for questions about the data or artifacts.