language:
- en
license: - cc-by-4.0
- apache-2.0
- mit
task_categories: - text-generation
configs: - config_name: default
data_files:- split: train
path: data/train.jsonl
- split: train
Dataset Description:
The Nemotron-SFT-Safety-v1 data is designed to align models to be robust against a variety of safety and security concerns that may arise in unaligned large language models.
This dataset is a collection of:
- A hybrid (open-source and synthetically generated) collection of prompts designed to elicit different model vulnerabilities, and
- Synthetically generated responses designed to steer model behavior towards safety-aligned values and enhance model robustness across a broad variety of potential attack vectors.
This dataset is ready for commercial use.
The prompts collection in this dataset comprise the following underlying subsets.
- Collected datasets:
- Nemotron Content Safety Dataset v2: Comprehensive coverage across a variety of content safety risks like self-harm, violence, criminal planning, profanity, and other harmful or harm-eliciting user requests.
- Gretel Safety Alignment Dataset v1: Supplemental content safety dataset with a similar purpose as Nemotron Content Safety Dataset.
- HarmfulTasks: Robustness against some templated but highly effective jailbreak attacks.
- RedTeam 2k: Supplemental robustness against handcrafted red-teaming prompts.
- Synthetically generated prompts:
sdg_oversafety: Contrastive safe prompts created by modifying unsafe prompts to maintain syntactic similarity but be semantically divergent.sdg_jailbreaks: Supplemental jailbreak robustness data.sdg_latent_injection: Improve resiliency against latent prompt injection attacks.sdg_sensitive_categories: Hard refusals for extreme harm categories like self-harm.sdg_bias_mitigation: Mitigate responses that may judge people based on characteristics like age.sdg_leakreplay: Mitigate attempts to elicit possibly copyrighted content.
The response generation techniques are carefully designed to elicit reasoning traces and assistant responses that are aligned with our safety behavior guides.
Dataset Owner(s):
NVIDIA Corporation
Dataset Creation Date:
Created on: 12/01/2025
Last Modified on: 12/01/2025
License/Terms of Use:
This dataset is governed by the Creative Commons Attribution 4.0 International License (CC BY 4.0).
Additional Information: Apache 2.0 License; MIT License.
Intended Usage:
This dataset is intended to be used by the community as part of their model's supervised finetuning (SFT) recipe to improve model alignment on various dimensions of safety and security, including:
- Appropriate responses to self-harm, violence, criminal planning, child sexual abuse material (CSAM), and other harmful or harm-eliciting user requests.
- Robustness to common jailbreak strategies, cybersecurity attacks, or red-teaming attempts designed to coerce a model into outputting disturbing content.
- Counter-acting tendency of models to become overly safe and refuse everything through contrastive hard negatives (requests that share syntactic structure but not semantic meaning with unsafe requests).
- Mitigating biased model outputs on axes like age-based discrimination.
- Mitigating leakage of potentially copyrighted content.
Dataset Characterization
Data Collection Method
- Hybrid: Human, Synthetic, Automated
Labeling Method
- Hybrid: Human, Synthetic, Automated
Dataset Format
Modality: Text
Format: JSONL
Structure: Text + Metadata
Dataset Quantification
Note: We source only the prompts from the following underlying datasets.
Any responses that may exist in an underlying dataset are discarded and regenerated using the response generation techniques described in the Dataset Description section.
| Prompts Collection | Samples |
|---|---|
Content safety (Nemotron Content Safety v2, Gretel Safety Alignment v1, sdg_sensitive_categories) |
25,536 |
Robustness against jailbreaks (HarmfulTasks, RedTeam 2k, sdg_jailbreaks, sdg_latent_injection) |
9,653 |
Robustness against over-refusals (sdg_oversafety) |
7,023 |
Robustness against demographic biases (sdg_bias_mitigation) |
2,000 |
Robustness against leaking sensitive content (sdg_leakreplay) |
933 |
| Total | 45,145 |
Total Disk Size: ~200MB
Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal developer teams to ensure this dataset meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns here