language:

• en
  pretty_name: "OR-Bench"
  viewer: true
  tags:
• llm
• alignment
• over-alignment
  license: "cc-by-4.0"
  dataset_info:
  • config_name: or-bench-80k
    features:
    • name: prompt
      dtype: string
    • name: category
      dtype: string
  • config_name: or-bench-hard-1k
    features:
    • name: prompt
      dtype: string
    • name: category
      dtype: string
  • config_name: or-bench-toxic
    features:
    • name: prompt
      dtype: string
    • name: category
      dtype: string

configs:

• config_name: or-bench-80k
  data_files:
  • split: train
    path: or-bench-80k.csv
• config_name: or-bench-hard-1k
  data_files:
  • split: train
    path: or-bench-hard-1k.csv
• config_name: or-bench-toxic
  data_files:
  • split: train
    path: or-bench-toxic.csv

task_categories:

• text-generation
• question-answering

OR-Bench: An Over-Refusal Benchmark for Large Language Models

Please see our leaderboard at HuggingFace Spaces.

Overall Plots of Model Performances

Below is the overall model performance. X axis shows the rejection rate on OR-Bench-Hard-1K and Y axis shows the rejection rate on OR-Bench-Toxic. The best aligned model should be on the top left corner of the plot where the model rejects the most number of toxic prompts and least number of safe prompts. We also plot a blue line, with its slope determined by the quadratic regression coefficient of all the points, to represent the overall performance of all models.

Overall Workflow

Below is the overall workflow of our pipeline. We automate the process of producing over-refusal prompts that is able to produce updated prompts constantly.

Detailed Model Performance

Here are the radar plots of selected model performances. The red area indicates the over-refusal rate and the blue area indicates the acceptance rate of toxic prompts. In both cases, the plotted area is the smaller the better.

Claude-2.1

Claude-2.1

Claude-3 Model Family

Claude-3-Haiku

Claude-3-Sonnet

Claude-3-Opus

Gemini Model Family

Gemma-7b

Gemma-2-27b

Gemini-1.0-pro

Gemini-1.5-flash

Gemini-1.5-pro

GPT-3.5-turbo Model Family

GPT-3.5-turbo-0301

GPT-3.5-turbo-0613

GPT-3.5-turbo-0125

GPT-4 Model Family

GPT-4-0125-preview

GPT-4o

GPT-4-1106-preview

Llama-2 Model Family

Llama-2-7b

Llama-2-13b

Llama-2-70b

Llama-3 Model Family

Llama-3-8b

Llama-3-70b

Mistral Model Family

Mistral-small-latest

Mistral-medium-latest

Mistral-large-latest

QWen Model Family

Qwen-1.5-7B

Qwen-1.5-32B

Qwen-1.5-72B