Evaluate just added

cais/wmdp

published by @cais

Safety benchmarks, over-refusal tests, and guard-model data.

task_categories:text-generationlanguage:enlicense:mitsize_categories:1K<n<10Kformat:parquetmodality:textlibrary:datasetslibrary:pandas +4 more
Lifetime downloads
738.4K
Last 30 days
30.1K
Likes
32
Size
1.4 MB

Description

Dataset Card for WMDP

The Weapons of Mass Destruction Proxy (WMDP) benchmark is a dataset of multiple-choice questions that serve as a proxy measurement of hazardous knowledge in biosecurity, cybersecurity, and chemical security. WMDP serves two roles: first, as an evaluation for hazardous knowledge in LLMs, and second, as a benchmark for unlearning methods to remove such hazardous knowledge.

See our paper, website, and GitHub for more details!

We implemented the WMDP evaluation… See the full description on the dataset page: https://huggingface.co/datasets/cais/wmdp.

Used by abliterated models none yet

No abliterated models in our catalog have declared this dataset in their YAML metadata yet. This can mean the dataset is used but not declared, is used outside abliteration workflows, or is new. The link will populate automatically as models are indexed.

Metadata

License
mit
Language
en
Created on HF
2024-03-05 · 2 years ago
Size category
1K<n<10K
Modalities
text
Formats
parquet
Libraries
datasetspandasmlcroissantpolars

Related datasets same stage · Evaluate

Read further

Total size
1.4 MB
Files
6
Formats
parquet, png
Head
7125571
Last updated on HF
2024-04-27

Download the dataset

Copy any of these snippets into your notebook or terminal. All three fetch directly from Hugging Face using your own credentials.

Recommended: datasets library (Python)
from datasets import load_dataset
ds = load_dataset("cais/wmdp")
Raw snapshot (Python)
from huggingface_hub import snapshot_download
snapshot_download(repo_id="cais/wmdp", repo_type="dataset")
git clone (requires git-lfs)
git clone https://huggingface.co/datasets/cais/wmdp

Files by folder from our last crawl

top level 3 files 294.1 KB
.gitattributes 2.3 KB file download
dataset.png 287.1 KB png download
README.md 4.6 KB md download
wmdp-bio/ 1 file 258.1 KB
test-00000-of-00001.parquet 258.1 KB parquet download
wmdp-chem/ 1 file 50.8 KB
test-00000-of-00001.parquet 50.8 KB parquet download
wmdp-cyber/ 1 file 805.7 KB
test-00000-of-00001.parquet 805.7 KB parquet download

No README stored for this dataset yet.

Download from HuggingFace

cais/wmdp

Catalog is the map. Apps are the tools.

Run models on your own machine, not in the cloud.

Every model page has an "Open in Abliteration" button that hands the model directly to the first-party desktop client, at the quantization your rig can actually run. No API keys, no subscription, no prompt leakage.

Open in Abliteration