license: cc-by-nc-sa-4.0
language:

• en
  size_categories:
• 1K<n<10K

Election Evaluations Dataset

Dataset Summary

This dataset includes some of the evaluations we implemented to assess language models' ability to handle election-related information accurately, harmlessly, and without engaging in persuasion targeting.

Dataset Description

The dataset consists of three CSV files, each focusing on a specific aspect of election-related evaluations:

1. eu_accuracy_questions.csv:

   • Contains information-seeking questions about European Union (EU) election-related topics.
   • The questions are designed to evaluate the accuracy of language models in providing factual information about EU elections.

2. harmlessness_eval.csv:

   • Includes a mix of harmless and potentially harmful questions on a range of topics related to United States (US) elections.
   • Each question is labeled as either "Harmless" or "Harmful."
   • The goal is to assess whether language models can identify and refuse to answer questions that are labeled as "Harmful," while providing appropriate responses to "Harmless" questions.

3. persuasion_targeting_eval.csv:

   • Contains questions related to various demographic groups within the United States in the context of elections.
   • Each question is labeled as either "Harmless" or "Harmful."
   • "Harmful" questions attempt to target specific populations for persuasion or manipulation.
   • The objective is to evaluate whether language models can identify and refuse to answer questions labeled as "Harmful", while providing appropriate responses to "Harmless" questions.

Disclaimer

Please note that these evaluations are generated by language models and may contain inaccuracies. While we conducted human evaluation on a subset of the dataset to assess quality, not all questions were manually reviewed. The dataset is intended for research purposes only and should not be considered a definitive assessment of language models' performance in handling election-related information.

Usage

from datasets import load_dataset
# Loading the data
dataset = load_dataset("Anthropic/election_questions")

Contact

For questions, you can email esin at anthropic dot com