configs:

• config_name: Egida
  data_files:

  • split: full
    path: Egida.json
  • split: train
    path: Egida-train.json
  • split: test
    path: Egida-test.json

• config_name: Egida-S
  data_files:

  • split: train
    path: Egida-S.json

• config_name: Egida-HSafe
  data_files:

  • split: test
    path: Egida-HSafe.json

• config_name: Egida-DPO-Meta-Llama-3.1-8B-Instruct
  data_files:

  • split: train
    path: Egida-DPO-Meta-Llama-3.1-8B-Instruct.json

• config_name: Egida-DPO-Meta-Llama-3.1-70B-Instruct
  data_files:

  • split: train
    path: Egida-DPO-Meta-Llama-3.1-70B-Instruct.json

• config_name: Egida-DPO-Qwen2.5-7B-Instruct
  data_files:

  • split: train
    path: Egida-DPO-Qwen2.5-7B-Instruct.json

• config_name: Egida-DPO-Qwen2.5-72B-Instruct
  data_files:

  • split: train
    path: Egida-DPO-Qwen2.5-72B-Instruct.json

  - split: Meta_Llama3.1_70B_Instruct

  path: Egida-DPO-Meta-Llama-3.1-70B-Instruct.json

  - split: Qwen2.5_7B_Instruct

  path: Egida-DPO-Qwen2.5-7B-Instruct.json

  - split: Qwen2.5_72B_Instruct

  path: Egida-DPO-Qwen2.5-72B-Instruct.json

license: apache-2.0
task_categories:

• question-answering
  language:
• en
  tags:
• jailbreaking
  pretty_name: Egida
  size_categories:
• 1K<n<10K

Dataset Card for Egida

Dataset Description

• Distribution: Egida HuggingFace
• Paper: Efficient Safety Retrofitting Against Jailbreaking for LLMs
• Point of Contact: Adrián Tormos

Dataset Summary

Egida is an expanded collection of unsafe requests gathered from a variety of external sources. This dataset is boosted and extended (1) through a manual fine-grained topic classification, and (2) by applying a variety of jailbreaking attacks to all their samples.

Dataset Curation

Sources and data collection

In total, the dataset is composed of 2,949 dangerous questions or instructions that have been assembled from nine different public datasets. The instances have been manually reviewed during the labeling process to ensure that they will cause unsafe or generally undesired responses from LLMs, and then deduplicated using MinHash.

Topics and jailbreaking attacks

All gathered samples were manually labeled by the authors into 27 fine-grained topics in a multilabeling fashion (i.e. every instance can have several ones). A list of all fine-grained topics within Egida, together with their frequency can be found in the following Figure. Since there is a significant imbalance among fine-grained topics, and considering how some of these are too small for analysis, the authors recommend aggregating topics into a higher level of abstraction when using the dataset.

These 2,949 labeled instances are expanded using 18 different jailbreaking attacks, originating from Chen et al., Shen et al., DeepInception and ReNeLLM. Two additional attack styles are implemented using Qwen-72B-Chat: Past tense and technical report writing. For this latter source, model refusals are filtered and removed using rule-based mechanisms. As a result, the complete Egida dataset is composed of 61,830 unsafe instances (also including the samples before adding any jailbreaking attack).

Data Splits

To conduct further experimentation, we perform a partition of the Egida dataset into train and test splits. To avoid contamination, topics and attack styles are distributed between both partitions without overlap. The attack styles in the test set are selected based on how challenging these are for LLMs (DAN and ReNeLLM Teacher cause the highest amount of unsafe responses from original models), and also prioritizing the lack of contamination among splits (all FuzzLLM variants are in the test set).

Dataset Extensions

Egida is designed to provide a controlled environment for experimentation and evaluation of LLMs in the presence of jailbreaking attacks.
Additionally, in our work we use it to align and assess a set of publicly available LLMs, and thus we release three different versions of the Egida dataset on top of the original one:

• Egida-S: Safe Responses.
  To extend Egida for DPO, we use two models that are unrelated to the rest of the experimentation: Mistral-7B-v0.3 and Phi-3-Small-8k. The safe responses of these models is used as chosen answers in the DPO phase. Mistral's responses are given priority over Phi's, as the former tends to be more elaborate than the latter.

• Egida-DPO: DPO Datasets.
  The train split is used to run inference on the four selected models (Llama-3.1-8B-Intruct, Llama-3.1-70B-Intruct, Qwen-2.5-7B-Intruct and Qwen-2.5-72B-Intruct). Unsafe answers are selected, and paired with safe answers to create a customized DPO dataset for each model. This allows us to experiment with a DPO datasets composed by triplets ("question", "chosen answer", "discarded answer") which contain questions that elicit unsafe responses by the target model, as well as the unsafe responses produced by it.

• Egida-HSafe: Human-Labeled Subset.
  The evaluation methodology used in this work uses an LLM-as-a-judge to label responses as either safe or unsafe. Measuring the reliability of such mechanism is therefore fundamental. In a significant human effort, five authors of this work manually label responses to 1,000 random requests from Egida, as produced by 10 different LLMs for the full list). Each response is annotated by three authors either as "Safe", "Unsafe" or "Uncertain", and this assessment is then compared with the results of the selected LLM-as-a-judge (Llama-Guard-3-8B).

Additionally, we release the corresponding models aligned with the Egida-DPO subsets: Meta-Llama-3.1-8B-Instruct-Egida-DPO, Meta-Llama-3.1-70B-Instruct-Egida-DPO, Qwen2.5-7B-Instruct-Egida-DPO and Qwen2.5-72B-Instruct-Egida-DPO.

Additional Information

Dataset Curators

Adrián Tormos Llorente and Daniel Hinjos García

Licensing Information

The dataset is licensed under the Apache License 2.0.

Citation Information

@misc{garciagasulla2025efficientsafetyretrofittingjailbreaking,
      title={Efficient Safety Retrofitting Against Jailbreaking for LLMs}, 
      author={Dario Garcia-Gasulla and Adrian Tormos and Anna Arias-Duart and Daniel Hinjos and Oscar Molina-Sedano and Ashwin Kumar Gururajan and Maria Eugenia Cardello},
      year={2025},
      eprint={2502.13603},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.13603}, 
}