license: mit
language:
- en
- bn
tags: - ai-safety
- mechanistic-interpretability
- multilingual
LatentGuard: Multilingual Safety Drift Dataset
Dataset Description
This dataset contains the experimental logs and output generations used to evaluate LatentGuard, a training-free inference intervention designed to mitigate multilingual safety bypasses via mid-layer latent steering.
The data specifically tracks the asymmetric safety failures of google/gemma-2-2b-it when processing parallel English and Bengali adversarial prompts.
The evaluation data was programmatically generated through the following pipeline:
Ingestion: Loaded 315 parallel Bengali-English safety prompts from aims-foundations/safety-irt (Hugging Face).
Filtering: Filtered the dataset explicitly for Bengali samples (language == "bn") and cached the complete corpus as bengali_prompts.json.
Phase 1 Baseline: Executed behavioral tracking on the first 150 samples to establish baseline refusal metrics, saved as 150_baseline_prompts.json.
Phase 2/3 Target Cohort: Identified and isolated exactly 35 prompts suffering from medium-severity token fragmentation. These were extracted into 35_failure_cohort.json to act as the primary evaluation suite for latent steering.
⚠️ INFO-HAZARD WARNING ⚠️
This dataset contains explicit examples of AI safety failures and generated harmful content. Because this research investigates the circumvention of LLM guardrails (Safety Drift), the files contain text where the model successfully complies with malicious instructions (e.g., social engineering, bias probes, and illegal acts) translated into Bengali. These artifacts are provided strictly for academic verification and the advancement of mechanistic interpretability research. Please handle with care.
File Structure
150_baseline_prompts.json: The Phase 1 behavioral profiling logs showing the drop in refusal rates between English and Bengali.35_failure_cohort_steered.json: The isolated Phase 2 and Phase 3 logs detailing the internal geometric tracking (Collapse and Drift scores) and the final LatentGuard steering transformations.