tags:
- cot-oracle
- chain-of-thought
- qa
- safety
- bias
- concept-labeled
license: mit
configs: - config_name: default
data_files:- split: train
path: data/train-*
dataset_info:
features: - name: corpus_id
dtype: string - name: task_family
dtype: string - name: task_type
dtype: string - name: prompt
dtype: string - name: target_response
dtype: string - name: answer_length
dtype: string
splits: - name: train
num_bytes: 662609
num_examples: 4196
download_size: 43782
dataset_size: 662609
- split: train
Concept CoT QA (Mini) — Ground Truth
4,196 QA pairs about chain-of-thought reasoning where ground truth comes directly from source dataset metadata. Zero LLM labeling cost.
QA Types
| Type | Count | Example Question | Answer Source |
|---|---|---|---|
| concept_domain | 496 | What domain is this reasoning about? | Keyword detection |
| concept_source | 496 | What dataset is this from? | Dataset name |
| is_harmful_request | 362 | Is the user's request harmful? | WildGuard/BeaverTails label |
| is_safe_content | 290 | Is the content safe? | Inverse of harmful label |
| is_cybercrime | 231 | Does this involve cybercrime? | Domain classifier |
| involves_violence | 231 | Does this involve violence? | Domain classifier |
| is_sycophancy_test | 56 | Is this a sycophancy test? | Anthropic dataset label |
| is_bias_test | 48 | Is this a bias test? | BBQ dataset label |
| bias_category | 48 | What type of bias? | BBQ category |
| is_jailbreak_attempt | 116 | Is this a jailbreak attempt? | Toxic Chat label |
| + 30 more types | ... | ... | ... |
Schema
| Field | Type | Description |
|---|---|---|
corpus_id |
str | Links to concept-cot-corpus-mini entry |
task_family |
str | Always concept_qa |
task_type |
str | Specific QA type |
prompt |
str | Activation-framed question |
target_response |
str | Ground-truth answer |
answer_length |
str | short/medium |
Usage
from datasets import load_dataset
ds = load_dataset("ceselder/concept-cot-qa-mini", split="train")
# Get all harmful-request detection QA
harm_qa = ds.filter(lambda x: x["task_type"] == "is_harmful_request")