task_categories:
- feature-extraction
- text-generation
language: - en
- zh
tags: - abliterate
- abliterated
- abliteration
- refusal
- harmful
- harmless
我使用本地模型将其翻译为中文，为了减少使用 "abliterator" 脚本时对llm中文能力的损害
Dataset for abliterating refusal in large language models
Contains "harmful" prompts where "is_harmful" field is true, and "harmless" prompts where false.
Credit: https://github.com/Sumandora/remove-refusals-with-transformers/
Source repo: https://huggingface.co/datasets/byroneverson/abliterate-refusal
Example usage:
import datasets
instructions = 512
dataset = load_dataset("lenML/abliterate-refusal-cn", split="train")
# Filter the dataset based on 'is_harmful'
harmful_dataset = dataset.filter(lambda x: x['is_harmful'] == True)
harmless_dataset = dataset.filter(lambda x: x['is_harmful'] == False)
# Randomly select 512 entries from each filtered dataset
harmful_en_instructions = random.sample(harmful_dataset['prompt'], instructions)
harmless_en_instructions = random.sample(harmless_dataset['prompt'], instructions)
# Randomly select 512 entries from each filtered dataset(chinese)
harmful_cn_instructions = random.sample(harmful_dataset['prompt_cn'], instructions)
harmless_cn_instructions = random.sample(harmless_dataset['prompt_cn'], instructions)