license: mit
language:
- yo
- ha
- ig
- sw
- ar
- en
tags: - safety
- mechanistic-interpretability
- activation-steering
- multilingual
- low-resource-languages
- refusal
- llm-safety
- african-languages
- jailbreak-benchmark
pretty_name: LSR Anchoring - Multilingual Benchmark & Safety Steering Results
size_categories: - n<1K
LSR Anchoring — Multilingual Benchmark & Safety Steering Results
This dataset contains all experimental results for two related studies:
Study 1 - Benchmark:
Multilingual Jailbreak Vulnerability Benchmark and Mitigation for Low-Resource African Languages
Study 2 — Mitigation:
Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining
Together they form a full pipeline: find the failure → characterise it geometrically → fix it, all without supervised African-language data.
Languages Covered
| Code | Language | Family | Resource Level |
|---|---|---|---|
yo |
Yoruba | Niger-Congo | Low |
ha |
Hausa | Afro-Asiatic | Low |
ig |
Igbo | Niger-Congo | Low |
igl |
Igala | Niger-Congo | Very Low (no ISO 639-1) |
sw |
Swahili | Bantu | Mid |
ar |
Arabic | Semitic | High |
Models Evaluated
| Model | Study | Method |
|---|---|---|
meta-llama/Meta-Llama-3-8B-Instruct |
Both | Benchmark + SAE & Mean-Act Steering |
meta-llama/Llama-3.1-70B-Instruct |
Steering only | Mean-Act Steering |
mistralai/Mistral-7B-Instruct-v0.3 |
Both | Benchmark + Mean-Act Steering |
Qwen/Qwen2.5-7B-Instruct |
Both | Benchmark + Mean-Act Steering |
google/gemma-2-9b-it |
Benchmark only | Benchmark |
Repository Structure
lsr-anchoring-phase2-results/
│
├── benchmark/ ← Study 1: Benchmark results
│ ├── raw_llama3-8b_*.csv — Per-prompt responses, Llama-3-8B
│ ├── raw_mistral-7b_*.csv — Per-prompt responses, Mistral-7B
│ ├── raw_qwen2.5-7b_*.csv — Per-prompt responses, Qwen2.5-7B
│ ├── raw_gemma2-9b_*.csv — Per-prompt responses, Gemma2-9B
│ ├── summary_llama3-8b_*.json — SRR summary by language
│ ├── summary_mistral-7b_*.json
│ ├── summary_qwen2.5-7b_*.json
│ ├── summary_gemma2-9b_*.json
│ ├── benchmark_results_merged.csv — All models merged
│ └── Model-EnglishSRR-AvgAfricanSRR-SafetyGap.csv
│
├── llama-3-8b-path-a/ ← Study 2: Llama-3.1 8B SAE steering
│ ├── results_rows.csv/json
│ ├── anchor_cache.json
│ ├── full_sweep.csv
│ └── logs/
│
├── llama-3-8b-path-b/ ← Study 2: Llama-3.1 8B Mean-Act steering
│ ├── results_8b_pathB_rows.csv/json
│ └── anchor_cache_8b_pathB.json
│
├── llama-3-70b/ ← Study 2: Llama-3.1 70B
│ ├── results_70b_rows.csv/json
│ ├── anchor_cache.json
│ └── logs/
│
├── mistral-7b/ ← Study 2: Mistral-7B steering
│ ├── results_mistral7b_rows.csv/json
│ ├── anchor_cache_mistral7b.json
│ └── logs/
│
├── qwen2.5-7b/ ← Study 2: Qwen2.5-7B steering
│ ├── results_qwen7b_rows.csv/json
│ ├── anchor_cache_qwen7b.json
│ └── logs/
│
└── data/ ← Study 2: Cross-model summary files
├── results_multilang_v3.json
├── results_phase2b_all.json
├── results_phase2b_arabic.json
└── results_phase2b_yoruba_ext.json
Study 1 — Benchmark Results Summary
Safety Refusal Rate (SRR) = refused harmful prompts / total harmful prompts.
Model,English,Yoruba,Hausa,Igbo,Igala,Swahili,Arabic
Llama-3-8B,99%,24%,24%,31%,21%,38%,87%
Mistral-7B,85%,11%,28%,15%,16%,25%,12%
Qwen2.5-7B,95%,4%,6%,14%,13%,0%,83%
Gemma2-9B,97%,55%,72%,20%,47%,94%,99%
Key finding: English SRR stays at 85–99% across all models. Every African language tested shows substantial collapse. Igala is the sharpest stress test, with weak refusal across three of four models despite no prior AI safety coverage.
Study 2 — LSR-Anchoring Results Summary
Model,Best Language,Peak SRR,KL at Peak,DPL at Peak
Llama-3.1 8B (SAE/Path A),Igala,0.844,0.31,0.06
Llama-3.1 8B (Mean-Act/Path B),Hausa,0.71,0.48,0.12
Llama-3.1 70B,"Yoruba, Igala",1.00,2.58–3.53,—
Mistral-7B,Igala,0.75,0.61,0.06
Qwen2.5-7B,Hausa,0.51,0.49,0.50
Notable finding: Arabic exhibits inverse transfer on every architecture. English-derived steering directions reduce refusal rates on Arabic harmful prompts as alpha increases (SRR → −0.20 at α=25 on 70B), indicating a geometric mismatch rather than a baseline effect. Do not apply LSR-Anchoring to Arabic-language agents without language-specific direction derivation.
Live Dashboard
Explore all benchmark and steering results interactively:
https://huggingface.co/spaces/low-resource-alignment-anon/lsr-dashboard
Citation
@misc{anon2026benchmark,
title = {Multilingual Jailbreak Vulnerability Benchmark and Mitigation for Low-Resource African Languages},
author = {Anonymous Authors},
year = {2026},
url = {[https://huggingface.co/datasets/low-resource-alignment-anon/lsr-anchoring-phase2-results](https://huggingface.co/datasets/low-resource-alignment-anon/lsr-anchoring-phase2-results)}
}
@misc{anon2026lsranchoring,
title = {Latent Space Refusal Anchoring for Low-Resource African Languages: Mechanistic Safety Recovery Without Retraining},
author = {Anonymous Authors},
year = {2026},
url = {[https://huggingface.co/datasets/low-resource-alignment-anon/lsr-anchoring-phase2-results](https://huggingface.co/datasets/low-resource-alignment-anon/lsr-anchoring-phase2-results)}
}
Code
All experiment scripts, benchmark code, and prompt sets are maintained in the anonymized repository organization interface.
License
MIT License.