configs:
- config_name: default
data_files:- split: train
path: "data/train.parquet"
- split: train
license: cc-by-nc-4.0
language:
- hi
- bn
- ta
- te
- mr
- gu
- kn
- ml
- pa
- or
- as
- ur
- sa
- en
task_categories: - text-generation
pretty_name: IndicSafetyBench
size_categories: - 1K<n<10K
IndicSafetyBench
Multi-turn safety evaluation benchmark for Indian languages. Tests models against 30 jailbreak techniques across 19 India-specific harm domains in 23 languages with 189 dialect varieties.
Stats
- 3,374 benchmark items
- 30 jailbreak techniques across 8 families
- 19 India harm domains (territorial, caste, religious, political, gender, etc.)
- 14 global harm categories (aligned with AILuminate v1.0 / Llama Guard 4)
- 23 languages (22 Indic + English)
- ~29% tactic chains (3-4 techniques combined in multi-turn)
Usage
from datasets import load_dataset
bench = load_dataset("anna-sarvam/indic-safety-eval", split="train")
Columns
| Column | Description |
|---|---|
item_id |
Unique identifier |
language, script |
Target language and script |
variety_id, variety_name |
Dialect variety |
technique, technique_family |
Jailbreak technique and family |
harm_category |
Global harm category (AILuminate-aligned) |
india_domain, india_topic |
India-specific harm domain and topic |
affected_groups |
Groups affected by the harm |
region, time_period |
Geographic and temporal context |
attack_angle |
How the adversarial prompt approaches the topic |
expected_behavior |
Expected safe model response |
num_turns |
Number of adversarial turns |
tactic_chain |
Whether this is a multi-technique chain |
turns |
List of adversarial user messages to replay |
Config Files
configs/taxonomy.yaml-- 6-field harm taxonomy definitionconfigs/techniques.yaml-- 30 jailbreak techniques with descriptions