dataset_info:
features:

• name: prompt
  dtype: string
• name: response
  dtype: string
• name: adversarial
  dtype: bool
• name: prompt_harm_label
  dtype: string
• name: response_refusal_agreement
  dtype: float64
• name: response_refusal_label
  dtype: string
• name: response_harm_label
  dtype: string
• name: subcategory
  dtype: string
• name: prompt_harm_agreement
  dtype: float64
• name: response_harm_agreement
  dtype: float64
• name: id
  dtype: int64
• name: language
  dtype: string
• name: prompt_label
  dtype: string
• name: prompt_categories
  dtype: string
• name: response_label
  dtype: string
• name: response_categories
  dtype: string
  splits:
• name: test
  num_bytes: 93846872
  num_examples: 29325
  download_size: 52852473
  dataset_size: 93846872
  configs:
• config_name: default
  data_files:
  • split: test
    path: data/test-*
    task_categories:
• text2text-generation
  language:
• ar
• zh
• cs
• nl
• en
• fr
• de
• hi
• th
• it
• ja
• ko
• pl
• pt
• ru
• es
• sv
  tags:
• safety
• multilingual
  size_categories:
• 10K<n<100K
  license: cc-by-4.0

PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages

Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations, and the corresponding training and evaluation datasets. PolyGuard is trained on PolyGuardMix, the largest multilingual safety training corpus to date containing 1.91M samples across 17 languages (e.g., Chinese, Czech, English, Hindi). We also introduce PolyGuardPrompts, a high quality multilingual benchmark with 29K samples for the evaluation of safety guardrails. Created by combining naturally occurring multilingual human-LLM interactions and human-verified machine translations of an English-only safety dataset (WildGuardMix; Han et al., 2024), our datasets contain prompt-output pairs with labels of prompt harmfulness, response harmfulness, and response refusal. Through extensive evaluations across multiple safety and toxicity benchmarks, we demonstrate that PolyGuard outperforms existing state-of-the-art open-weight and commercial safety classifiers by 5.5%. Our contributions advance efforts toward safer multilingual LLMs for all global users.

Languages

The data supports 17 languages and are reported in the table below.

Data Fields

• prompt: user prompt input by user
• response: model's response to the user prompt
• prompt_harm_label: if the prompt is harmful
• response_refusal_label: if the model refuses the user's request
• response_harm_label: if the response is harmful
• prompt_safety_categories: list of violated safety categories by harmful prompt
• response_safety_categories: list of violated safety categories by harmful response
• metadata: language and source of data sample

Citation

@misc{kumar2025polyguardmultilingualsafetymoderation,
      title={PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages}, 
      author={Priyanshu Kumar and Devansh Jain and Akhila Yerukola and Liwei Jiang and Himanshu Beniwal and Thomas Hartvigsen and Maarten Sap},
      year={2025},
      eprint={2504.04377},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2504.04377}, 
}